System for negotiating at least two sets of video capabilities between two nodes to perform video conferencing between the nodes according to the selected set

ABSTRACT

Nodes of a computer-based teleconferencing network may be able to encode and decode video signals using different video compression/decompression algorithms with different parameters. In order to conduct a video teleconference between two (or more) such nodes, the nodes must first agree on what video algorithms to use. In the present invention, video capabilities are negotiated between nodes, where each node uses a table that identify one or more sets of video capabilities for that node. After video capabilities have been successfully negotiated, the video conferencing can proceed using the negotiated capabilities. Negotiated video capabilities may include algorithms, frame rate, frame resolution, bit rate, and bitstream format.

INCORPORATION BY REFERENCE AND RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.08/341,402, filed Nov. 16, 1994, now U.S. Pat. No. 5,524,110, which is acontinuation-in-part of U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned, which is a continuation-in-part ofU.S. patent application Ser. No. 08/157,694, filed Nov. 24, 1993, nowU.S. Pat. No. 5,506,954, all three of which are incorporated herein intheir entireties by reference.

This application is related to U.S. patent application Ser. No.08/342,076 (filed Nov. 16, 1994, now abandoned), U.S. patent applicationSer. No. 08/305,206 (filed Sep. 13, 1994, now U.S. Pat. No. 5,600,684),U.S. patent application Ser. No. 08/137,319 (filed Oct. 14, 1993, nowU.S. Pat. No. 5,452,299), U.S. patent application Ser. No. 08/170,146(filed Dec. 20, 1993, now U.S. Pat. No. 5,581,702), U.S. patentapplication Ser. No. 08/235,955 (filed Apr. 28, 1994, now U.S. Pat. No.5,493,514), and U.S. patent application Ser. No. 08/133,612 (filed Oct.12, 1993, now U.S. Pat. No. 5,410,698), which are all incorporatedherein in their entireties by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to audio/video conferencing, and, inparticular, to systems for real-time audio, video, and data conferencingin windowed environments on personal computer systems.

2. Description of the Related Art

It is desirable to provide real-time audio, video, and data conferencingbetween personal computer (PC) systems operating in windowedenvironments such as those provided by versions of Microsoft® Windows™operating system. There are difficulties, however, with providingreal-time conferencing in non-real-time windowed environments. It isalso desirable to provide conferencing between PC systems over two ormore different transports.

It is accordingly an object of this invention to overcome thedisadvantages and drawbacks of the known art and to provide real-timeaudio, video, and data conferencing between PC systems operating innon-real-time windowed environments over two or more differenttransports.

It is a particular object of the present invention to provide real-timeaudio, video, and data conferencing between PC systems operating under aMicrosoft® Windows™ operating system over ISDN and LAN networks.

Further objects and advantages of this invention will become apparentfrom the detailed description of a preferred embodiment which follows.

SUMMARY OF THE INVENTION

The present invention comprises a computer-implemented process andapparatus for video conferencing. According to a preferred embodiment,video capabilities are negotiated by a first node with a second node,wherein the first node bases its negotiating on a first-node table thatidentifies one or more sets of video capabilities for the first node.One of the sets of video capabilities for the first node is selected forvideo conferencing between the first and second nodes. Local encodedvideo signals are generated by the first node in accordance with theselected set of video capabilities. The local encoded video signals aretransmitted from the first node to the second node. Remote encoded videosignals are received by the first node from the second node. The remoteencoded video signals are decoded by the first node in accordance withthe selected set of video capabilities.

DESCRIPTION OF THE DRAWINGS

Other objects, features, and advantages of the present invention willbecome more fully apparent from the following detailed description, theappended claims, and the accompanying drawings in which:

FIG. 1 is a block diagram representing real-time point-to-point audio,video, and data conferencing between two PC systems;

FIG. 2 is a block diagram of the hardware configuration of theconferencing system of each PC system of FIG. 1;

FIG. 3 is a block diagram of the hardware configuration of the videoboard of the conferencing system of FIG. 2;

FIG. 4 is a block diagram of the hardware configuration of theaudio/comm (ISDN) board of the conferencing system of FIG. 2;

FIG. 5 is a block diagram of the software configuration of theconferencing system of each PC system of FIG. 1;

FIG. 6 is a block diagram of the hardware configuration of theaudio/comm (ISDN) board of FIG. 4;

FIG. 7 is a block diagram of the conferencing interface layer betweenthe conferencing applications of FIG. 5, on one side, and the comm,video, and audio managers of FIG. 5, on the other side;

FIG. 8 is a representation of the conferencing call finite state machine(FSM) for a conferencing session between a local conferencing system(i.e., caller) and a remote conferencing system (i.e., callee);

FIG. 9 is a representation of the conferencing stream FSM for eachconferencing system participating in a conferencing session;

FIG. 10 is a representation of the video FSM for the local video streamand the remote video stream of a conferencing system during aconferencing session;

FIG. 11 is a block diagram of the software components of the videomanager of the conferencing system of FIG. 5;

FIG. 12 is a representation of a sequence of N walking key frames;

FIG. 13 is a representation of the audio FSM for the local audio streamand the remote audio stream of a conferencing system during aconferencing session;

FIG. 14 is a block diagram of the architecture of the audio subsystem ofthe conferencing system of FIG. 5;

FIG. 15 is a block diagram of the interface between the audio task ofFIG. 5 and the audio hardware of audio/comm (ISDN) board of FIG. 2;

FIG. 16 is a block diagram of the interface between the audio task andthe comm task of FIG. 5;

FIG. 17 is a block diagram of the comm subsystem of the conferencingsystem of FIG. 5;

FIG. 18 is a block diagram of the comm subsystem architecture for twoconferencing systems of FIG. 5 participating in a conferencing sessionover an ISDN connection;

FIG. 19 is a representation of the comm subsystem application FSM for aconferencing session between a local site and a remote site;

FIG. 20 is a representation of the comm subsystem connection FSM for aconferencing session between a local site and a remote site;

FIG. 21 is a representation of the comm subsystem control channelhandshake FSM for a conferencing session between a local site and aremote site;

FIG. 22 is a representation of the comm subsystem channel establishmentFSM for a conferencing session between a local site and a remote site;

FIG. 23 is a representation of the comm subsystem processing for atypical conferencing session between a caller and a callee;

FIG. 24 is a representation of the structure of a video packet as sentto or received from the comm subsystem of the conferencing system ofFIG. 5;

FIG. 25 is a representation of the compressed video bitstream for theconferencing system of FIG. 5;,

FIG. 26 is a representation of a compressed audio packet for theconferencing system of FIG. 5;

FIG. 27 is a representation of the reliable transport comm packetstructure;

FIG. 28 is a representation of the unreliable transport comm packetstructure;

FIG. 29 are diagrams indicating typical TII-DLM connection setup andteardown sequences;

FIGS. 30 and 31 are diagrams of the architecture of the audio/comm(ISDN) board;

FIG. 32 is a diagram of the audio/comm (ISDN) board environment;

FIG. 33 is a flow diagram of the on-demand application invocationprocessing of the conferencing system of FIG. 5;

FIG. 34 is a flow diagram of an example of the processing implementedwithin the conferencing system of FIG. 5 to manage two conferencingapplications in a single conferencing session with a remote conferencingsystem;

FIG. 35 represents the flow of bits between two remote high-resolutioncounters used to maintain clock values over a conferencing network;

FIG. 36 is a flow diagram of the processing of the conferencing systemof FIG. 1 to control the flow of signals over reliable channels;

FIG. 37 is a flow diagram of the preemptive priority-based transmissionprocessing implemented by the communications subsystem of theconferencing system of FIG. 1;

FIG. 38 is a state diagram for the complete rate negotiation processing;

FIG. 39 is a state diagram for the rate negotiation processing for acalled node during a 64 KBPS upgrade;

FIG. 40 is a state diagram for the rate negotiation processing for acalling node during a 64 KBPS upgrade; and

FIG. 41 is a state diagram for the rate negotiation processing inloopback mode during a 64 KBPS upgrade;

FIG. 42 is a flow diagram of the processing by the conferencing systemof FIGS. 5 and 17 during the automatic transport detection implementedat install time;

FIG. 43 is a block diagram showing the network connections made by theconferencing system of FIGS. 5 and 17 during the automatic transportdetection implemented at run time;

FIG. 44 is a representation of the DLMLAN packet header format;

FIG. 45 is a representation of the MDM packet header format for LANtransmissions;

FIG. 46 is a representation of the connection messages for a typicalconferencing session from the perspective of the MDMs on the local andremote nodes;

FIG. 47 is a flow diagram of the video negotiation processing betweentwo conferencing systems of FIG. 1;

FIG. 48 is a flow diagram of the call-progress processing when theplacement of a conference call is successful;

FIG. 49 is a representation of the interrupt-time processing forreceiving data signals by the audio/video conferencing system of FIG. 5;

FIG. 50 is a representation of the interrupt-time processing fortransmitting data signals by the audio/video conferencing system of FIG.5;

FIG. 51 is a representation of the auto registration environment forvideo conferencing;

FIG. 52 is a representation of the architecture for auto registrationand remote confidence testing for the new node of FIG. 51;

FIG. 53 is a flow diagram of the processing for the auto registrationand remote confidence testing of the auto registration environment ofFIG. 51;

FIG. 54 is a flow diagram of the processing implemented by the client(i.e., a new node) for the auto registration processing of FIG. 53;

FIG. 55 is a flow diagram of the processing implemented by a confidencetest server for the auto registration processing of FIG. 53;

FIG. 56 is a representation of the auto registration file format; and

FIG. 57 are connection diagrams that show the interactions between a DLMand an MDM in connection and session establishment and tear-down.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

Point-To-Point Conferencing Network

Referring now to FIG. 1, there is shown a block diagram representingreal-time point-to-point audio, video, and data conferencing between twoPC systems, according to a preferred embodiment of the presentinvention. Each PC system has a conferencing system 100, a camera 102, amicrophone 104, a monitor 106, and a speaker 108. The conferencingsystems communicate via network 110, which may be either an integratedservices digital network (ISDN), a local area network (LAN), or a widearea network (WAN). Each conferencing system 100 receives, digitizes,and compresses the analog video signals generated by camera 102 and theanalog audio signals generated by microphone 104. The compressed digitalvideo and audio signals are transmitted to the other conferencing systemvia network 110, where they are decompressed and converted for play onmonitor 106 and speaker 108, respectively. In addition, eachconferencing system 100 may generate and transmit data signals to theother conferencing system 100 for play on monitor 106. The video anddata signals are displayed in different windows on monitor 106. Eachconferencing system 100 may also display the locally generated videosignals in a separate window.

Camera 102 may be any suitable camera for generating NSTC or PAL analogvideo signals. Microphone 104 may be any suitable microphone forgenerating analog audio signals. Monitor 106 may be any suitable monitorfor displaying video and graphics images and is preferably a VGAmonitor. Speaker 108 may be any suitable device for playing analog audiosignals and is preferably a headset.

Conferencing System Hardware Configuration

Referring now to FIG. 2, there is shown a block diagram of the hardwareconfiguration of each conferencing system 100 of FIG. 1. Eachconferencing system 100 comprises host processor 202, video board 204,audio/comm (ISDN) board 206, LAN board 210, and ISA bus 208.

Referring now to FIG. 3, there is shown a block diagram of the hardwareconfiguration of video board 204 of FIG. 2. Video board 204 comprisesindustry standard architecture (ISA) bus interface 310, video bus 312,pixel processor 302, video random access memory (VRAM) device 304, videocapture module 306, and video analog-to-digital (A/D) converter 308.

Referring now to FIG. 4, there is shown a block diagram of the hardwareconfiguration of audio/comm (ISDN) board 206 of FIG. 2. Audio/comm(ISDN) board 206 comprises ISDN interface 402, memory 404, digitalsignal processor (DSP) 406, and ISA bus interface 408, audioinput/output (I/O) hardware 410.

LAN board 210 of FIG. 2 may be any conventional LAN card that supportsstandard driver interfaces and is preferably an Intel® EtherExpress™ 16CLAN Combo Card.

Conferencing System Software Configuration

Referring now to FIG. 5, there is shown a block diagram of the softwareconfiguration each conferencing system 100 of FIG. 1. Video microcode530 resides and runs on pixel processor 302 of video board 204 of FIG.3. Comm task 540 and audio task 538 reside and run on DSP 406 ofaudio/comm (ISDN) board 206 of FIG. 4. The one or more network stacks560 reside and run partially on host processor 202 of FIG. 2 andpartially on LAN board 210 of FIG. 2. All of the other software modulesdepicted in FIG. 5 reside and run on host processor 202.

Video, Audio, and Data Processing

Referring now to FIGS. 3, 4, and 5, audio/video conferencing application502 running on host processor 202 provides the top-level local controlof audio and video conferencing between a local conferencing system(i.e., local site or endpoint) and a remote conferencing system (i.e.,remote site or endpoint). Audio/video conferencing application 502controls local audio and video processing and establishes links with theremote site for transmitting and receiving audio and video over the ISDNor LAN network 110. Similarly, data conferencing application 504, alsorunning on host processor 202, provides the top-level local control ofdata conferencing between the local and remote sites. Conferencingapplications 502 and 504 communicate with the audio, video, and commsubsystems using conference manager 544, conferencing applicationprogramming interface (API) 506, LAN management interface (LMI) API 556,LMI manager 558, video API 508, comm API 510, and audio API 512. Thefunctions of conferencing applications 502 and 504 and the APIs they useare described in further detail later in this specification.

Audio Processing

During conferencing, audio I/O hardware 410 of audio/comm (ISDN) board206 digitizes analog audio signals received from microphone 104 andstores the resulting uncompressed digital audio to memory 404 via ISAbus interface 408. Audio task 538, running on DSP 406, controls thecompression of the uncompressed audio and stores the resultingcompressed audio back to memory 404.

Audio Processing for ISDN-Based Processing

For ISDN-based conferencing, comm task 540, also running on DSP 406,formats the locally-generated compressed audio for ISDN transmission andtransmits the compressed ISDN-formatted audio to ISDN interface 402 fortransmission to the remote site over ISDN network 110.

During ISDN-based conferencing, ISDN interface 402 also receives fromISDN network 110 compressed ISDN-formatted audio generated by the remotesite and stores the compressed ISDN-formatted audio to memory 404. Commtask 540 then reconstructs the compressed audio format and stores thecompressed audio back to memory 404. Audio task 538 controls thedecompression of the compressed audio and stores the resultingdecompressed audio back to memory 404. ISA bus interface then transmitsthe decompressed audio to audio I/O hardware 410, whichdigital-to-analog (D/A) converts the decompressed audio and transmitsthe resulting analog audio signals to speaker 108 for play.

Thus, for ISDN-based conferencing, audio capture/compression anddecompression/playback are performed entirely within audio/comm (ISDN)board 206 without going through the host processor. As a result, audiois continuously played during an ISDN-based conferencing sessionregardless of what other applications are running on host processor 202.

Audio Processing for LAN-Based Processing

For LAN-based conferencing, audio task 538 passes the locally-generatedcompressed audio to the audio manager 520, which sends the compressedaudio via comm API 510 to the comm manager 518 for transmission by thenetwork stack 560 to the remote site via the LAN network 110.

During LAN-based conferencing, the network stack 560 also receives fromLAN network 110 compressed LAN-formatted audio generated by the remotesite and passes the compressed LAN-formatted audio to comm manager 518.Comm manager 518 then reconstructs the compressed audio format andpasses the compressed audio via audio API 512 to audio manager 520,which stores the compressed audio into memory 404 of the audio/comm(ISDN) board 206 of FIG. 4. As in ISDN-based conferencing, audio task538 controls the decompression of the compressed audio and stores theresulting decompressed audio back to memory 404. ISA bus interface thentransmits the decompressed audio to audio I/O hardware 410, whichdigital-to-analog (D/A) converts the decompressed audio and transmitsthe resulting analog audio signals to speaker 108 for play.

Video Processing

Concurrent with the audio processing, video A/D converter 308 of videoboard 204 digitizes analog video signals received from camera 102 andtransmits the resulting digitized video to video capture module 306.Video capture module 306 decodes the digitized video into YUV colorcomponents and delivers uncompressed digital video bitmaps to VRAM 304via video bus 312. Video microcode 530, running on pixel processor 302,compresses the uncompressed video bitmaps and stores the resultingcompressed video back to VRAM 304. ISA bus interface 310 then transmitsvia ISA bus 208 the compressed video to video/host interface 526 runningon host processor 202.

Video/host interface 526 passes the compressed video to video manager516 via video capture driver 522. Video manager 516 calls audio manager520 using audio API 512 for synchronization information. Video manager516 then time-stamps the video for synchronization with the audio. Videomanager 516 passes the time-stamped compressed video to comm manager 518via comm API 510.

Video Processing for ISDN-Based Conferencing

For ISDN-based conferencing, comm manager 518 passes thelocally-generated compressed video through digital signal processing(DSP) interface 528 to ISA bus interface 408 of audio/comm (ISDN) board206, which stores the compressed video to memory 404. Comm task 540 thenformats the compressed video for ISDN transmission and transmits theISDN-formatted compressed video to ISDN interface 402 for transmissionto the remote site over ISDN network 110.

During ISDN-based conferencing, ISDN interface 402 also receives fromISDN network 110 ISDN-formatted compressed video generated by the remotesite system and stores the ISDN-formatted compressed video to memory404. Comm task 540 reconstructs the compressed video format and storesthe resulting compressed video back to memory 404. ISA bus interfacethen transmits the compressed video to comm manager 518 via ISA bus 208and DSP interface 528. Comm manager 518 passes the compressed video tovideo manager 516 via video API 508. Video manager 516 passes thecompressed video to video decode driver 548 for decompressionprocessing. Video decode driver 548 passes the decompressed video tovideo playback driver 550, which formats the decompressed video fortransmission to the graphics device interface (GDI) (not shown) of theMicrosoft® Windows™ operating system for eventual display in a videowindow on monitor 106.

Video Processing for LAN-Based Conferencing

For LAN-based conferencing, comm manager 518 formats thelocally-generated compressed video for LAN transmission and transmitsthe LAN-formatted compressed video to the network stack 560 fortransmission to the remote site over LAN network 110.

During LAN-based conferencing, the network stack 560 also receives fromLAN network 110 LAN-formatted compressed video generated by the remotesite system and passes the LAN-formatted compressed video to commmanager 518. Comm manager 518 then reconstructs the compressed videoformat and passes the compressed video via video API 508 to videomanager 516. As in ISDN-based conferencing, video manager 516 passes thecompressed video to video decode driver 548 for decompressionprocessing. Video decode driver 548 passes the decompressed video tovideo playback driver 550, which formats the decompressed video fortransmission to the graphics device interface (GDI) (not shown) of theMicrosoft® Windows™ operating system for eventual display in a videowindow on monitor 106.

Data Processing

For data conferencing, concurrent with audio and video conferencing,data conferencing application 504 generates and passes data to commmanager 518 using conferencing API 506 and comm API 510.

Data Processing for ISDN-Based Conferencing

For ISDN-based conferencing, comm manager 518 passes thelocally-generated data through board DSP interface 532 to ISA businterface 408, which stores the data to memory 404. Comm task 540formats the data for ISDN transmission and stores the ISDN-formatteddata back to memory 404. ISDN interface 402 then transmits theISDN-formatted data to the remote site over ISDN network 110.

During ISDN-based conferencing, ISDN interface 402 also receives fromISDN network 110 ISDN-formatted data generated by the remote site andstores the ISDN-formatted data to memory 404. Comm task 540 reconstructsthe data format and stores the resulting data back to memory 404. ISAbus interface 408 then transmits the data to comm manager 518, via ISAbus 208 and DSP interface 528. Comm manager 518 passes the data to dataconferencing application 504 using comm API 510 and conferencing API506. Data conferencing application 504 processes the data and transmitsthe processed data to Microsoft® Windows™ GDI (not shown) for display ina data window on monitor 106.

Data Processing for LAN-Based Conferencing

For LAN-based conferencing, comm manager 518 formats thelocally-generated data for LAN transmission and transmits theLAN-formatted data video to the network stack 560 for transmission tothe remote site over LAN network 110.

During LAN-based conferencing, the network stack 560 also receives fromLAN network 110 LAN-formatted data generated by the remote site systemand passes the LAN-formatted data to comm manager 518. Comm manager 518then reconstructs the data and passes the data to data conferencingapplication 504 using comm API 510 and conferencing API 506. As inISDN-based conferencing, data conferencing application 504 processes thedata and transmits the processed data to Microsoft® Windows™ GDI (notshown) for display in a data window on monitor 106.

Hardware Configuration for Conferencing System

LAN board 210 of FIG. 2 may be any suitable board for transmitting andreceiving digital packets over a local (or wide) area network and ispreferably an Intel® EtherExpress™ 16 card with appropriate control andnetwork protocol software. Conferencing system 100 is capable ofsupporting LAN-based conferencing under different LAN transportstandards (e.g., Novell IPX, Internet User Datagram Protocol (UDP),and/or NetBIOS standards). Furthermore, conferencing system 100 iscapable of supporting LAN-based conferencing with different LAN productsfor a single LAN transport standard (e.g., LAN WorkPlace (LWPUDP) byNovell and FTPUDP by FTP Software, Inc., both of which conform to theLAN UDP standard). Thus, LAN board 210 corresponds to the LAN transportsthat are supported in conferencing system 100. Those skilled in the artwill understand that more than one network stack 560 may be used tointerface with a single LAN board 210.

Referring now to FIG. 6, there is shown a block diagram of the hardwareconfiguration of audio/comm (ISDN) board 206 of FIG. 4. Referring now toFIGS. 30 and 31, there are shown diagrams of the architecture of theaudio/comm (ISDN) board 206. Referring now to FIG. 32, there is shown adiagram of the audio/comm (ISDN) board environment. The description forthe rest of this section is the same as the description for the sectionof the same name in U.S. patent application Ser. No. 08/157,694.

Software Architecture for Conferencing System

The software architecture of conferencing system 100 of FIGS. 2 and 5has three layers of abstraction. A computer supported collaboration(CSC) infrastructure layer comprises the hardware (i.e., video board204, audio/comm (ISDN) board 206, and LAN board 210) and host/boarddriver software (i.e., video/host interface 526, DSP interface 528, andnetwork stack 560) to support video, audio, and comm, as well as theencode method for video (running on video board 204) and encode/decodemethods for audio (running on audio/comm (ISDN) board 206). Thecapabilities of the CSC infrastructure are provided to the upper layeras a device driver interface (DDI).

A CSC system software layer provides services for instantiating andcontrolling the video and audio streams, synchronizing the two streams,and establishing and gracefully ending a call and associatedcommunication channels. This functionality is provided in an applicationprogramming interface (API). This API comprises the extended audio andvideo interfaces and the communications APIs (i.e., conference manager544, conferencing API (VCI) 506, LAN management interface (LMI) API 556,LMI manager 558, video API 508, video manager 516, video capture driver522, video decode driver 548, video playback driver 550, comm API 510,comm manager 518, Wave API 514, Wave driver 524, PWave API 552, audioAPI 512, and audio manager 520).

A CSC applications layer brings CSC to the desktop. The CSC applicationsmay include video annotation to video mail, video answering machine,audio/video/data conferencing (i.e., audio/video conferencingapplication 502 and data conferencing application 504), and groupdecision support systems.

Audio/video conferencing application 502 and data conferencingapplication 504 rely on conference manager 544 and conferencing API 506,which in turn rely upon video API 508, comm API 510, and audio API 512to interface with video manager 516, comm manager 518, and audio manager520, respectively. Comm API 510 and comm manager 518 provide atransport-independent interface (TII) that provides communicationsservices to conferencing applications 502 and 504. The communicationssoftware of conferencing system 100 may be designed to support differenttransport mechanisms, such as ISDN, SW56, and LAN (e.g., SPX/IPX,TCP/IP, or NetBIOS). The TII isolates the conferencing applications fromthe underlying transport layer (i.e., transport-medium-specific DSPinterface 528). The TII hides the network/connectivity specificoperations. In conferencing system 100, the TII hides the ISDN and LANlayers. The DSP interface 528 is hidden in a datalink module (DLM). TheLAN interface is hidden within a media dependent module (MDM). The TIIprovides services to the conferencing applications for openingcommunication channels (within the same session) and dynamicallymanaging the bandwidth. The bandwidth is managed through a transmissionpriority scheme.

In an embodiment in which conferencing system 100 performs softwarevideo decoding, AVI capture driver 522 is implemented on top ofvideo/host interface 526 (the video driver). In an alternativeembodiment in which conferencing system 100 performs hardware videodecoding, an AVI display driver is also implemented on top of video/hostinterface 526.

The software architecture of conferencing system 100 comprises threemajor subsystems: video, audio, and communication. The audio and videosubsystems are decoupled and treated as "data types" (similar to text orgraphics) with conventional operations like open, save, edit, anddisplay. The video and audio services are available to the applicationsthrough video-management and audio-management extended interfaces,respectively.

Conferencing system 100 is implemented mostly in the C++ computerlanguage using the Microsoft® Foundation Classes (MFC) with portionsimplemented in the C7.0 computer language.

Audio/Video Conferencing Application

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CMIF.LIB

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CCm

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Loading and Unloading

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Registering and Unregistering

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Call Support

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Channel Pair Support

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Stream Support

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CMDLL Callback

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

NO VCI Support

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Miscellaneous

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CImageSize

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CImageState

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

PSVIDEO.EXE

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Frame, View, and Image

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Class Descriptions

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CCyApp

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CCyFrameWnd

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CCyAppFrame

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CVideoFrame

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CVideoController

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Auto-Sizing of Video Windows

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Split and Combined Modes

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Control Channel Management

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Mute Message

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

High-Quality Snapshot Message

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Application Launch

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Application Launch Response

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CChanPair

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Video View Class Relationships

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Handset Class Relationships

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Dialog Boxes

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Helper Classes

Dialog Helper

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Fast Bitmap Buttons

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Data Conferencing Application

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Conference Manager

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Conference Manager Overview

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Implementation Details

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned. Additional information on theconference manager API is found in APPENDIX A of this specification.

Conference Application Installation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Conference Application Registration

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

VCI Call Handler Callback

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Channel Pair Establishment

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Critical Sections

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Call Notification and Caller ID

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Audible Call Progress

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

On Demand Application Invocation

Referring now to FIG. 33, there is shown a flow diagram of the on-demandapplication invocation processing of conferencing system 100 of FIG. 5.The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Managing Multiple Applications

Referring now to FIG. 34, there is shown a flow diagram of an example ofthe processing implemented within conferencing system 100 of FIG. 5 tomanage two conferencing applications in a single conferencing sessionwith a remote conferencing system. The description for this section isthe same as the description for the section of the same name in U.S.patent application Ser. No. 08/340,172, filed Nov. 15, 1994, nowabandoned.

Conferencing API

Referring now to FIG. 7, there is shown a block diagram of conferencemanager 544 and conferencing API 506 between conferencing applications502 and 504, on one side, and comm API 508, LMI API 556, video API 510,and audio API 512, on the other side. The description for this sectionis the same as the description for the section of the same name in U.S.patent application Ser. No. 08/340,172, filed Nov. 15, 1994, nowabandoned. Additional information on the conferencing API is found inAPPENDIX B of this specification.

Interfacing with the Comm Subsystem

Conferencing API 506 supports the following comm services with the commsubsystem:

Comm initialization--initialize a session in the comm subsystem on whichthe call will be made.

Call establishment--place a call to start a conference.

Channel establishment--establish two comm channels for videoconferencing control information, two comm channels for audio(incoming/outgoing), four comm channels for video (incoming data andcontrol and outgoing data and control).

Call termination--hang up a call and close all active channels.

Comm Initialization/Uninitialization

Initialization of a session in the comm subsystem on which a call may bemade by the user of conferencing system A of FIG. 1 and the user ofconferencing system B of FIG. 1 is implemented as follows:

Conferencing APIs A and B call LMI₋₋ AddLANTransport to initialize theirLAN management interface (LMI) subsystems.

Conferencing APIs A and B receive a LMI₋₋ ADDTRANS₋₋ RESPONSE callbackfrom the LMI subsystem.

Conferencing APIs A and B call BeginSession to initialize their commsubsystems.

Conferencing APIs A and B receive a SESS₋₋ BEGIN callback from the commsubsystem.

Conferencing APIs A and B then notify the conferencing applications witha CFM₋₋ INIT₋₋ TRANSP₋₋ NTFY callback.

Uninitialization of a session in the comm subsystem is implemented asfollows:

Conferencing APIs A and B call LMI₋₋ DeleteLANTransport to uninitializetheir LAN management interface (LMI) subsystems.

Conferencing APIs A and B receive a LMI₋₋ DELTRANS₋₋ RESPONSE callbackfrom the LMI subsystem.

Conferencing APIs A and B call EndSession to uninitialize their commsubsystems.

Conferencing APIs A and B receive a SESS₋₋ CLOSED callback from the commsubsystem.

Conferencing APIs A and B then notify the conferencing applications witha CFM₋₋ UNINIT TRANSP₋₋ NTFY callback.

Call Establishment

Establishment of a call between the user of conferencing system A ofFIG. 1 and the user of conferencing system B of FIG. 1 is implemented asfollows:

Conferencing API A calls LMI₋₋ RequestPermission to request permissionto make the conference call from the management computer.

Conferencing API A receives a LMI₋₋ PERM₋₋ RESPONSE callback from theLMI subsystem. If permission is denied, conferencing API A notifies theconferencing application with a CFM₋₋ REJECT₋₋ NTFY callback. Ifpermission is granted, establishment of the call is continued.

Conferencing API A calls LMI₋₋ CallCommit to indicate to LMI that thecall will be made.

Conferencing API A calls MakeConnection to dial conferencing API B'snumber.

Conferencing API B receives a CONN₋₋ REQUESTED callback from the commsubsystem.

Conferencing API B calls LMI₋₋ RequestPermission to request permissionto accept the conference call from the management computer.

Conferencing API B receives a LMI₋₋ PERM₋₋ RESPONSE callback from theLMI subsystem. If permission is denied, conferencing API B rejects thecall with RejectConnection, and notifies the conferencing applicationwith a CFM₋₋ DENIAL₋₋ NTFY callback. If permission is granted,establishment of the call is continued.

Conferencing API B sends the call notification to the graphic userinterface (GUI) with a CFM₋₋ CALL₋₋ NTFY callback; and, if user Baccepts the call via the GUI, conferencing API B proceeds with thefollowing steps.

Conferencing API B calls LMI₋₋ CallCommit to indicate to LMI that thecall will be accepted.

Conferencing API B calls AcceptConnection to accept the incoming callfrom conferencing API A.

Conferencing APIs A and B receive CONN₋₋ ACCEPTED callback from the commsubsystem.

Conferencing API A calls OpenChannel to open its outgoing conferencingcontrol channel.

Conferencing API B receives the CHAN₋₋ REQUESTED callback for theincoming control channel and accepts it via AcceptChannel. Thenconferencing API B calls OpenChannel to open its outgoing conferencingcontrol channel.

Conferencing API A receives the CHAN₋₋ ACCEPTED callback for itsoutgoing control channel and calls RegisterChanHandler to receivechannel callbacks from the comm subsystem. Then conferencing API Areceives the CHAN₋₋ REQUESTED callback for the incoming control channeland accepts it via AcceptChannel.

Conferencing API B receives the CHAN₋₋ ACCEPTED callback for itsoutgoing control channel and calls RegisterChanHandler to receivechannel callbacks from the comm subsystem.

Conferencing API A sends a Login Request on the control channel, whichconferencing API B receives.

Conferencing API B sends a Login Response on the control channel, whichconferencing API A receives.

Conferencing APIs A and B negotiate conference capabilities betweenthemselves. Capabilities that are negotiated include: negotiationprotocol version, audio compression algorithm, video compressionalgorithm, video frame rate, video capture resolution, video bitrate,and data sharing capabilities.

Conferencing API A sends a Capabilities Request on the control channel,specifying conference requirements, which conferencing API B receives.

Conferencing API B sends a Capabilities Response on the control channel,accepting or modifying conference requirements, which conferencing API Areceives.

When conferencing APIs A and B agree upon conference capabilities, thecapabilities are saved and will be communicated to the application viathe CFM₋₋ ACCEPT₋₋ NTFY callback.

Conferencing API A calls OpenChannel to open its outgoing audio channel.

Conferencing API B receives the CHAN₋₋ REQUESTED callback for theincoming audio channel and accepts it via AcceptChannel.

Conferencing API A receives the CHAN₋₋ ACCEPTED callback for theoutgoing audio channel.

The last three steps are repeated for the video data channel and thevideo control channel.

Conferencing API B then turns around and repeats the above 4 steps(i.e., opens its outbound channels for audio/video data/video control).

Conferencing API A sends Participant Information on the control channel,which conferencing API B receives.

Conferencing API B sends Participant Information on the control channel,which conferencing API A receives.

Conferencing APIs A and B call LMI₋₋ ConferenceCommit to indicate to LMIthat the conference is in progress.

Conferencing APIs A and B then notify the conferencing applications witha CFM₋₋ ACCEPT₋₋ NTFY callback.

Channel Establishment

Video and audio channel establishment is implicitly done as part of callestablishment, as described above, and need not be repeated here. Forestablishing other channels such as data conferencing, the conferencingAPI passes through the request to the comm manager, and sends the commmanager's callback to the user's channel manager.

Call Termination

Termination of a call between users A and B is implemented as follows(assuming user A hangs up):

Conferencing API A unlinks local/remote video/audio streams from thenetwork.

Conferencing API A calls LMI₋₋ ConferenceLeave to indicate to LMI thatthe conference is being closed.

Conferencing API A then calls the comm subsystem's CloseConnection.

The comm subsystem implicitly closes all channels, and sends CHAN₋₋CLOSED callbacks to the conferencing API A.

Conferencing API A closes its remote audio/video streams on receipt ofthe CHAN₋₋ CLOSED callback for its inbound audio/video channels,respectively.

Conferencing API A then receives the CONN₋₋ CLOSE₋₋ RESP callback afterthe call is cleaned up completely. Conferencing API A notifies itsconferencing application with a CFM₋₋ HANGUP₋₋ NTFY callback.

In the meantime, conferencing API B would have received the CHAN₋₋CLOSED callbacks from the comm subsystem for all the closed channels.

Conferencing API B closes its remote audio/video streams on receipt ofthe CHAN₋₋ CLOSED callback for its inbound audio/video channels,respectively.

Conferencing API B unlinks its local audio/video streams from thenetwork on receipt of the CHAN₋₋ CLOSED callback for its outboundaudio/video channels, respectively.

Conferencing API B then receives a CONN₋₋ CLOSED callback from the commsubsystem.

Conferencing API B calls LMI₋₋ ConferenceLeave to indicate to LMI thatthe conference is being closed.

Conferencing API B then notifies its conferencing application with aCFM₋₋ HANGUP₋₋ NTFY callback.

Interfacing with the Audio and Video Subsystems

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Capture/Monitor/Transmit Local Streams

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Receive/Play Remote Streams

The description for this section is the same as the descriptions for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Control Local/Remote Streams

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Snap an Image from Local Video Streams

Referring now to FIG. 8, there is shown a representation of theconferencing call finite state machine (FSM) for a conferencing sessionbetween a local conferencing system (i.e., caller) and a remoteconferencing system (i.e., callee). Referring now to FIG. 9, there isshown a representation of the conferencing stream FSM for eachconferencing system participating in a conferencing session. Thedescription for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned. Differences include changes to theCF₋₋ Init function and new functions CF₋₋ Uninit, CF₋₋ InitTransport,CF₋₋ UninitTransport, and CF₋₋ ChangeTransportMaxVideoBitrate, asfollows:

CF₋₋ Init Initializes the LAN management interface (LMI), audio andvideo subsystems, and initializes data structures required forconferencing.

CF₋₋ Uninit Uninitializes the LMI, audio, and video subsystems. If aconference call is in progress, it is gracefully destroyed.

CF₋₋ InitTransport Initializes a LAN or ISDN transport stack so thatconference calls may be made or received on a particular transport type.The maximum video bitrate allowed on this transport is specified.

CF₋₋ UninitTransport Uninitializes a transport stack, so calls may nolonger be made or received on a particular transport type.

CF₋₋ ChangeTransportMaxVideoBitrate Changes the maximum video bitrateallowed on a transport.

These functions are defined in further detail later in thisspecification in APPENDIX B.

In addition, conferencing API 506 supports the following additionalmessages returned to conferencing applications 502 and 504 from thevideo, comm, and audio subsystems in response to some of theabove-listed functions:

CFM₋₋ INIT₋₋ TRANSP₋₋ NTFY Indicates that transport stack initializationhas completed successfully or unsuccessfully.

CFM₋₋ UNINIT₋₋ TRANSP₋₋ NTFY Indicates that transport stackuninitialization has completed.

CFM₋₋ UNFNIT₋₋ NTFY Indicates that the conferencing API subsystemuninitialization has completed.

CFM₋₋ DENIAL₋₋ NTFY Indicates that a call request initiated from theremote site has been received, but the local site was denied permissionto accept the call by the management computer.

CFM₋₋ ERROR₋₋ NTFY Indicates that an error has occurred in the commsubsystem.

CFM₋₋ KILL₋₋ NTFY Indicates that the management computer has demandedthe conference call be terminated.

Video Subsystem

The video subsystem of conferencing system 100 of FIG. 5 comprises videoAPI 508, video manager 516, video decode driver 548, video playbackdriver 550, video capture driver 522, and video/host interface 526running on host processor 202 of FIG. 2 and video microcode 530 runningon video board 204.

In an embodiment of the invention of U.S. patent application Ser. No.08/157,694 (filed Nov. 24, 1993), the video subsystem encoded anddecoded video according to a single video compression technique, that,for purposes of this patent application, may be referred to as theISDN-rate video (IRV) technique. The video processing and videobitstream format described in defined in U.S. patent application Ser.No. 08/157,694, now U.S. Pat. No. 5,506,954, corresponded to that IRVtechnique.

The video subsystem of the present invention, however, is capable ofencoding and decoding video according to more than one video compressiontechnique. In one embodiment, the video system is capable of encodingand decoding video using both the IRV technique and a multi-rate video(MRV) technique. The following sections of this specification referprimarily to the IRV technique. The MRV technique is described infurther detail in later sections of this specification starting with thesection entitled "Compressed Multi-Rate Video Bitstream."

Video API

Referring now to FIG. 10, there is shown a representation of the videoFSM for the local video stream and the remote video stream of aconferencing system during a conferencing session. The description forthis section is the same as the description for the section of the samename in U.S. patent application Ser. No. 08/157,694, now U.S. Pat. No.5,506,954. Additional information on the video API is found in APPENDIXC of this specification.

Video Manager

Referring now to FIG. 11, there is shown a block diagram of the softwarecomponents of video manager (VM) 516 of FIG. 5. The description for thissection is the same as the description for the section of the same namein U.S. patent application Ser. No. 08/157,694, now U.S. Pat. No.5,506,954.

Capture/Playback Video Effects

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Video Stream Restart

Referring now to FIG. 12, there is shown a representation of a sequenceof N walking key frames. The description for this section is the same asthe description for the section of the same name in U.S. patentapplication Ser. No. 08/157,694, now U.S. Pat. No. 5,506,954.

Audio/Video Synchronization

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Alternative Timestamp Driver

FIG. 35 represents the flow of bits between two remote high-resolutioncounters used to maintain clock values over a conferencing network. Thedescription for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Bit Rate Throttling

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Multiple Video Formats

The description for this section is the same as the description, for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Normal Display Resolution

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Quarter Display Resolution

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Video Frame Format/Capture Interface

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Playback Implementation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Self-Calibration

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Measurement

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

File-Based Capture (File Playback)

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Playback Statistics

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

VCost Function

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

VM DLL

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

VCapt EXE

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

VPlay EXE

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Palette Creation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Extra RealizePalette Logic

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Netw DLL

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

AVSync DLL

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Video Capture Driver

Video capture driver 522 of FIG. 5 follows driver specifications setforth in the Microsoft® Video for Windows™ (VfW) Developer Kitdocumentation. This documentation specifies a series of applicationprogram interfaces (APIs) to which video capture driver 522 responds.Microsoft® Video for Windows™ (VfW) is a Microsoft® extension to theMicrosoft® Windows™ operating system. VfW provides a common framework tointegrate audio and video into an application program. Video capturedriver 522 extends the basic Microsoft® API definitions by providingnine "custom" APIs that provide direct control of enhancements to thestandard VfW specification to enable and control bit rate throttling andlocal video monitoring. Video capture driver 522 captures images in the"raw" YVU9 format and compresses them using either the IRV or the MRVcompression technique. Video capture driver 522 controls bit ratethrottling and local video monitoring differently for IRV and MRVcompression.

Bit rate throttling controls the bit rate of a transmitted videoconference data stream. Bit rate throttling is based on the quality ofthe captured video image and the image capture frame rate. Ahigh-quality image has more fine detail information than a low-qualityimage. A user of conferencing system 100 is able to vary the relativeimportance of image quality and frame capture rate with a custom capturedriver API.

The data bandwidth capacity of the video conference communicationchannel is fixed. The amount of captured video data to be transmitted isvariable, depending upon the amount of motion that is present in thevideo image. The capture driver is able to control the amount of datathat is captured by changing the quality of the next captured videoframe and by not capturing the next video frame (i.e., "dropping" theframe).

The image quality is determined on a frame-by-frame basis using thefollowing equation: ##EQU1## Quality is the relative image quality ofthe next captured frame. A lower Quality number represents a lower imagequality (less image detail). TargetSize is the desired size of acaptured and compressed frame. TargetSize is based on a fixed, desiredcapture frame rate.

Normally, video capture driver 522 captures new video frames at a fixed,periodic rate which is set by the audio/video conference applicationprogram. Video capture driver 522 keeps a running total of the availablecommunication channel bandwidth. When video capture driver 522 is readyto capture the next video frame, it first checks the available channelbandwidth and if there is insufficient bandwidth (due to a large,previously captured frame), then video capture driver 522 delayscapturing the next video frame until sufficient bandwidth is available.Finally, the size of the captured video frame is subtracted from theavailable channel bandwidth total.

A user of conferencing system 100 may control the relationship betweenreduced image quality and dropped frames by setting image qualitycharacteristics. For IRV compression, the user may set a minimum imagequality value which controls the range of permitted image qualities,from a wide range down to a narrow range of only the best imagequalities. For MRV compression, the user may set image quality usingthree parameters: motion estimation, spatial filtering, and temporalfiltering. The effects of these parameters on image quality arediscussed in U.S. patent application Ser. No. 08/235,955 (filed Apr. 28,1994).

Bit rate throttling is implemented inside of the video capture driverand is controlled by the following VfW extension APIs:

CUSTOM₋₋ SET₋₋ DATA₋₋ RATE Sets the data rate of the communicationschannel.

CUSTOM₋₋ SET₋₋ FPS Sets the desired capture frame rate.

CUSTOM₋₋ SET₋₋ QUAL₋₋ PERCENT Sets the minimum image quality value (IRVonly).

CUSTOM₋₋ SET₋₋ MOTION₋₋ EST Enables or disables motion estimation (MRVonly).

CUSTOM₋₋ SET₋₋ SPATIAL₋₋ FILT Enables or disables spatial filtering (MRVonly).

CUSTOM₋₋ SET₋₋ TEMPORAL₋₋ FILT Sets the level of temporal filtering (MRVonly).

The local video monitoring extension to VfW gives the video capturedriver 522 the ability to output simultaneously both a compressed and anon-compressed image data stream to the application, while remainingfully compatible with the Microsoft® VfW interface specification.Without this capability, audio/video conferencing application 502 wouldhave to decompress and display the image stream generated by the capturedriver in order to provide local video monitoring, which would place anadditional burden on the host processor and may decrease the frameupdate rate of the displayed image.

According to the VfW interface specification, the compressed image datais placed in an output buffer. When local video monitoring is active, anuncompressed copy of the same image frame is appended to the outputbuffer immediately following the compressed image data. The capturedriver generates control information associated with the output buffer.This control information reflects only the compressed image block of theoutput buffer and does not indicate the presence of the uncompressedimage block, making local video monitoring fully compatible with otherVfW applications. A reserved, 32-bit data word in the VfW controlinformation block indicates to a local video monitor aware applicationthat there is a valid uncompressed video image block in the outputbuffer. The application program may then read and directly display theuncompressed video image block from the output buffer.

For the IRV technique, the uncompressed image data may be in eitherDevice Independent Bitmap (DIB) or YUV9 format. For the MRV technique,the YVU9 format is used for the uncompressed image data. DIB formatimages are a fixed size, whereas YUV9 format images may be increased insize while retaining image quality. For both IRV and MRV techniques, theYUV9 images are converted into DIB format by the video display driverbefore they are displayed on the computer monitor.

The capture driver allows the uncompressed video image to be capturedeither normally or mirrored (reversed left to right). In normal mode,the local video monitoring image appears as it is viewed by a videocamera--printing appears correctly in the displayed image. In mirroredmode, the local video monitoring image appears as if it were beingviewed in a mirror.

The CUSTOM₋₋ SET₋₋ DIB₋₋ CONTROL extension API controls the local videomonitoring capabilities of the video capture driver.

Custom APIs for Video Capture Driver

The CUSTOM₋₋ SET₋₋ FPS message sets the frame rate for a video capture.This message is used while in streaming capture mode.

The CUSTOM₋₋ SET₋₋ KEY message informs the capture driver to produce onekey frame as soon as possible. The capture driver will typically produceone delta frame before the key frame. Once the key frame has beenencoded, delta frames will typically follow.

The CUSTOM₋₋ SET₋₋ DATA₋₋ RATE message informs the capture driver to setan output data rate. This data rate value is in KBits per second andtypically corresponds to the data rate of the communications channelover which the compressed video data will be transmitted.

The CUSTOM₋₋ SETQUAL₋₋ PERCENT message controls the relationship betweenreducing the image quality and dropping video frames when the IRVcompressed video data stream size exceeds the data rate set by theCUSTOM₋₋ SET₋₋ DATA₋₋ RATE message. For example, a CUSTOM₋₋ SET₋₋ QUAL₋₋PERCENT value of 0 means that the driver should reduce the image qualityas much as possible before dropping frames and a value of 100 means thatvideo frames should be dropped before the image quality is lowered. TheCUSTOM₋₋ SET₋₋ QUAL₋₋ PERCENT message is used only with IRV compression.

The CUSTOM₋₋ SET₋₋ DIB₋₋ CONTROL message controls the uncompressed DIBor YUV9 format image output. With IRV compression, the uncompressedimage may be in DIB format at either (80×60) or (160×120) pixelresolution or may be in YVU9 format at (160×120) resolution. With MRVcompression, only the (160×120) YVU9 image format is supported. Allimages are available in either mirrored (reversed left to right) or anormal image. This API controls the following four parameters:

Uncompressed image enable/disable

Mirrored/normal image

The uncompressed image size

Image data format (DIB or YVU9)

The default condition is for the uncompressed image to be disabled. Onceset, these control flags remain in effect until changed by anotherCUSTOM₋₋ SET₋₋ DIB₋₋ CONTROL message. The uncompressed image data isappended to the video data buffer immediately following the compressedimage data. The uncompressed DIB or YUV9 data have the bottom scan-linedata first and the top scan-line data last in the buffer.

The CUSTOM₋₋ SET₋₋ VIDEO message controls the video demodulatorCONTRAST, BRIGHTNESS, HUE (TINT), and SATURATION parameters. These videoparameters are also set by the capture driver at initialization and viaa video control dialog box.

The CUSTOM₋₋ SET₋₋ MOTION₋₋ EST message allows MRV motion estimation tobe enabled or disabled to improve image quality. This message is usedonly with MRV compression.

The CUSTOM₋₋ SET₋₋ SPATIAL₋₋ FILT message allows MRV spatial filteringto be enabled or disabled to improve image quality. This message is usedonly with MRV compression.

The CUSTOM₋₋ SET₋₋ TEMPORAL₋₋ FILT message allows the MRV temporalfilter strength to be altered to improve image quality. This message isused only with MRV compression.

Video Microcode

The video microcode 530 of FIG. 5 running on video board 204 of FIG. 2performs video compression. The preferred video compression techniquesare disclosed in later sections of this specification starting with thesection entitled "Compressed Video Bitstream."

Audio Subsystem

Referring now to FIG. 13, there is shown a block diagram of thearchitecture of the audio subsystem. The description for this section isthe same as the description for the section of the same name in U.S.patent application Ser. No. 08/157,694, now U.S. Pat. No. 5,506,954. Inaddition, referring again to FIG. 13, if the network connection is overa LAN, then the audio task 538 on System A sends the packetized,time-stamped audio data to the commstub task 1308, which sends it to theaudio manager 520 on the host processor 202. The audio manager 520passes the data to TII 510 for delivery to the remote system. The audiodata from System B is delivered by TII 510 to the audio manager 520 onSystem A (on the host). The audio manager 520 sends the packet to thecommstub task 1308 which passes it on to the audio task 538.

Audio API

Referring now to FIG. 14, there is shown a representation of the audioFSM for the local audio stream and the remote audio stream of aconferencing system during a conferencing session. The description forthis section is the same as the description for the section of the samename in U.S. patent application Ser. No. 08/157,694, now U.S. Pat. No.5,506,954. Additional information on the audio API is found in APPENDIXD of this specification.

Audio Manager

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Audio Manager Device Driver Interface

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,except for the following. The expected messages (generated by aMicrosoft® OpenDriver SDK call to installable device drivers) and thedrivers response are as follows:

DRV₋₋ LOAD Reads any configuration parameters associated with thedriver. Allocates any memory required for execution. This call is onlymade the first time the driver is opened.

DRV₋₋ ENABLE Ensures that an audio/comm board is installed andfunctional. For audio/comm board 206 of FIG. 2, this means the DSPinterface 532 is accessible. This call is only made the first time thedriver is opened.

DRV₋₋ OPEN This call is made each time OpenDriver is called. The audiomanager can be opened once for input, once for output (i.e., it supportsone full duplex conversation), and any number of times for devicecapabilities query. This call allocates the per application data. Thisincludes information such as the callback and the application instancedata and buffers for transferring audio between the host and the audioboard for LAN connections. If this is an input or output call, it startsthe DSP audio task and sets up communication between host and DSP audiotask (e.g. setup mail boxes, register callbacks). If this is the firstopen of an input or output stream, it starts the commstub task.

The installable device driver will respond to the close protocolmessages defined by Microsoft®. The expected messages (generated by theMicrosoft® SDK CloseDriver call to installable device drivers) and thedrivers response are as follows:

    ______________________________________                                        DRV.sub.-- CLOSE                                                                        Frees the per application data allocated                                      in DRV.sub.-- OPEN message.                                         DRV.sub.-- DISABLE                                                                      Ignored.                                                            DRV.sub.-- FREE                                                                         Ignored.                                                            ______________________________________                                    

This call sequence is symmetric with respect to the call sequencegenerated by OpenDriver. It has the same characteristics and behavior asthe open sequence does. Namely, it receives one to three messages fromthe CloseDriver call dependent on the driver's state and it generatesone callback per CloseDriver call. Three messages are received when thedriver's final instance is being closed. Only the DRV₋₋ CLOSE message isgenerated for other CloseDriver calls.

DRV₋₋ CLOSE message closes the audio thread that corresponds to theaudio stream indicated by HASTRM. The response to the close message isin response to a message sent back from the board indicating that thedriver has closed. Therefore, this call is asynchronous.

AM₋₋ LINKIN Message

The AM₋₋ LINKIN message is sent to the driver whenever the audio managerfunction ALinkIn is called. Param1 is a pointer to the followingstructure:

    ______________________________________                                                typedef struct.sub.-- ALinkStruct {                                             BOOL     ToLink;                                                              CHANID   ChanId;                                                              } ALinkStruct, FAR * 1pALinkStruct;                                 ______________________________________                                    

ToLink contains a BOOL value that indicates whether the stream is beinglinked in or unlinked (TRUE is linked in and FALSE is unlinked). If noerror is detected and ToLink is TRUE, the channel and the playbackstream should be linked together. The driver calls TII to determinewhether the transport associated with the channel is ISDN. If so, thedriver calls TII to determine the ID of the channel on the boardassociated with the TII channel ID. It then sends the Audio Task theALINKIN₋₋ TMSG message with the board channel ID as a parameter. Thiscauses the Audio Task to link up with the specified comm channel andbegin playing incoming audio. If the transport associated with thechannel is not ISDN, the driver prepares to receive data from thespecified TII channel and send the data to the commstub task. It thensends the Audio Task the ALINKIN₋₋ HOST₋₋ TMSG. This causes the AudioTask to link up with the commstub task to receive the audio data andplay it.

Breaking the link between the audio stream handle and the channel ID isdone when the ToLink field is set to FALSE. The audio manager sends theALINKIN₋₋ TMSG to the task along with the channel ID. The Audio Taskresponds to this message by unlinking the specified channel ID (i.e., itdoes not play any more audio).

Errors that the host task will detect are as follows:

The channel ID does not represents a valid read stream.

The audio stream handle is already linked or unlinked (detected onhost).

The audio stream handle is not a playback handle.

If those or any interface errors (e.g. message pending) are detected thecallback associated with this stream is notified immediately. If noerrors are detected, the ALINKIN₋₋ TMSG or ALINKIN₋₋ HOST₋₋ TMSG isissued to the DSP interface and the message pending flag is set for thisstream. Upon receiving the callback for this message, the callbackassociated with this stream is made, and finally the message pendingflag is unset.

AM₋₋ LINKOUT Message

The AM₋₋ LINKOUT message is sent to the driver whenever the audiomanager function ALinkOut is called. Param1 is a pointer to thefollowing structure:

    ______________________________________                                                typedef struct.sub.-- ALinkStruct {                                             BOOL     ToLink;                                                              CHANID   ChanId;                                                              } ALinkStruct, FAR * 1pALinkStruct;                                 ______________________________________                                    

ToLink contains a BOOL value that indicates whether the stream is beinglinked out or unlinked (TRUE is linked out and FALSE is unlinked). If noerror is detected and ToLink is TRUE, the channel and the audio instream should be linked together. The driver calls TII to determinewhether the transport associated with the channel is ISDN. If so, thedriver calls TII to determine the ID of the channel on the boardassociated with the TII channel ID. It then sends the Audio Task theALINKOUT₋₋ TMSG message with the board channel ID as a parameter. Thiscauses the Audio Task to link up with the specified comm channel andsend it captured audio. If the transport associated with the channel isnot ISDN, the driver prepares to receive data from the commstub task andsend it to the specified TII channel. It then sends the Audio Task theALINKOUT₋₋ HOST₋₋ TMSG. This causes the Audio Task to link up with thecommstub task to send it captured audio data.

Breaking the link between the audio stream handle and the channel ID isdone when ToLink field is set to FALSE. The audio manager will send theALINKOUT₋₋ TMSG to the task along with the channel ID. The Audio Taskwill respond to this message by unlinking the specified channel ID (i.e.it won't send any more audio).

Errors that the host task will detect are as follows:

The channel ID does not represents a valid write stream.

The audio stream handle is already linked or unlinked (detected onhost).

The audio stream handle is not a audio in handle.

If those or any interface errors (e.g., message pending) are detected,the callback associated with this stream is notified immediately. If noerrors are detected, the ALINKOUT₋₋ TMSG or ALINKOUT₋₋ HOST₋₋ TMSG isissued to the DSP interface and the message pending flag is set for thisstream. Upon receiving the callback for this message, the callbackassociated with this stream is made, and finally the message pendingflag is unset.

Audio Manager Interface with the DSP Interface

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Host Processor to Audio/Comm Board Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,except for the following:

ALINKIN₋₋ TMSG: Connects/disconnects the audio task with a virtualcircuit supported by the network task. The local and remote channel IDs(valid on the board) are passed to the audio task in the first twoDWORDs of the dwArgs array. The flag specifying whether to link orunlink is passed in the third DWORD.

ALINKIN₋₋ HOST₋₋ TMSG: Connects/disconnects the audio task with thecommstub task to receive audio to the host. The flag specifying whetherto link or unlink is passed to the audio task in the third DWORD of thedwArgs array. The first two DWORDS are ignored.

ALINKOUT₋₋ TMSG: Connects the audio task with a virtual circuitsupported by the network task. The local and remote channel IDs (validon the board) are passed to the audio task in the first two DWORDs ofthe dwArgs array. The flag specifying whether to link or unlink ispassed in the third DWORD.

ALINKOUT₋₋ HOST₋₋ TMSG: Connects the audio task with a virtual circuitsupported by the network task. The flag specifying whether to link orunlink is passed to the audio task in the third DWORD of the dwArgsarray. The first two DWORDS are ignored.

Audio/Comm Board to Host Processor Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Wave Audio Implementation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Audio Subsystem Audio/Comm (ISDN) Board-Resident Implementation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954. In addition, the audio task 538 of FIG. 13connects with the commstub task 1308. This interface allows the audiotask to exchange compressed data packets of audio samples with the host202, which is responsible for delivering them to the remote system whenthe network is not ISDN (e.g., LAN). As the name implies, this task is astandin for the comm task. The interface is the same as that between theaudio task 538 and the comm task 540.

Audio Task Interface with Host Device Driver

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Audio Task Interface with Audio Hardware

Referring now to FIG. 15, there is shown a block diagram of interfacebetween the audio task 538 and the audio hardware of audio/comm (ISDN)board 206 of FIG. 13. The description for this section is the same asthe description for the section of the same name in U.S. patentapplication Ser. No. 08/157,694, now U.S. Pat. No. 5,506,954.

Timestamp Driver

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

(De)Compression Drivers

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Mixer/Splitter Driver

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Mixer Internal Operation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Echo Suppression Driver

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Spectral Equalization

In one embodiment of the present invention, the microphone 104 andspeaker 108 of a conferencing node of FIG. 1 are part of a singleearpiece component, such as an Enterprise™ headset sold by Plantronics.Because the microphone is located away from the mouth and in physicalcontact with the user's head near the ear, the audio signals may becomedistorted. These distortions may be due to reverberation signals thatreflect off the user's cheek, sounds from the user's mouth that becomeout of phase at the microphone, and/or the directionality/loss of thehigher frequencies. These distortions may combine with artifacts of theaudio coder to degrade the quality of the audio portion of aconferencing session.

Digital filtering is applied to the audio signals to attempt to correctfor the distortions that result from using a combined microphone/speakerearpiece. When using the Plantronics Enterprise™ microphone, the digitalfilter is implemented using a cascade of a second-order high-passChebyshev Type I Infinite Impulse Response filter and a sixth-orderInfinite Impulse Response filter designed using the Steiglitzapproximation, which produces a 3 dB bump at 2 kHz to enhanceperception.

This digital filtering is implemented as part of the equalizer stackabledriver 1514 in the capture side audio processing as shown in FIG. 15.The equalizer driver 1514 can be selectively enabled or disabled. Whenthe user selects a combined earpiece headset, then the equalizer driver1514 is enabled and each audio frame is digitally filtered before beingpassed to the next driver on the audio stack (i.e., echo/suppressionstackable driver 1512 of FIG. 15). When the user selects anotherconfiguration of microphone and speaker (e.g., a speakerphone or adirectional boom microphone headset), then the equalizer driver 1514 isdisabled and each audio frame is passed on to the echo/suppressiondriver 1512 without any processing. The equalizer driver 1514 isimplemented as a driver under the Spectron Microsystems SPOX™ operatingsystem.

Audio Task Interface with Comm Task

Referring again to FIG. 13, the audio task 538 sends and receives audiopackets from either the comm task 540 or the commstub task 1308,depending on whether the network connection is over ISDN or LAN. Theinterface the audio task uses in the same in either case. Throughoutthis section, references to comm task 540 also apply to commstub task1308.

The interface between the audio task to the audio hardware is based onSPOX streams. Unfortunately, SPOX streams connect tasks to source andsink device drivers, not to each other. Audio data are contained withinSPOX array objects and associated with streams. To avoid unnecessarybuffer copies, array objects are passed back and forth between the command audio subsystems running on the audio/comm board using SPOX streamsand a pipe driver. The actual pipe driver used will be based on a SPOXdriver called NULLDEV. Like Spectron's version, this driver simplyredirects buffers it receives as an IO₋₋ SINK to the IO₋₋ SOURCE stream;no buffer copying is performed. Unlike Spectron's pipe driver, however,NULLDEV does not block the receiving task if no buffers are availablefrom the sending stream and discards buffers received from the IO₋₋SOURCE stream if no task has made the IO₋₋ SINK stream connection to thedriver. In addition, NULLDEV will not block or return errors to thesender. If no free buffers are available for exchange with the sender'slive buffer, NULLDEV returns a previously queued live buffer. Thisaction simulates a dropped packet condition.

Setup and teardown of these pipes will be managed by a message protocolbetween the comm task and audio task threads utilizing the existing TMBmailbox architecture built into the Mikado DSP interface. The interfaceassumes that the comm task or commstub task is running, a networkconnection has been established, and channel ID's (i.e., virtual circuitID's) have been allocated to the audio subsystem by the conferencingAPI. The interface requires the comm task and commstub task each to makeavailable to the audio threads the handle to its local mailbox TMB₋₋MYMBOX. This is the mailbox a task uses to receive messages from thehost processor. The mailbox handle is copied to a global memory locationand retrieved by the threads using the global data package discussedlater in this specification. The audio task chooses which mailbox touse, and thus whether to communicate with the comm task or the commstubtask, based on which message it receives from the host. ALINKOUT₋₋ TMSGand ALINKIN₋₋ TMSG cause it to use the comm task mailbox, and ALINKOUT₋₋HOST₋₋ TMSG and ALINKIN₋₋ HOST₋₋ TMSG cause ti to use the commstub taskmailbox. In the case of an ISDN connection, the audio task becomes thechannel handler for the audio channels. Otherwise, the audio driver onthe host becomes the channel handler.

Message Protocol

Referring now to FIG. 16, there is shown a block diagram of theinterface between the audio task 538 and the comm task 540 of FIGS. 5and 13. The description for this section is the same as the descriptionfor the section of the same name in U.S. patent application Ser. No.08/157,694, which applies to conferencing over an ISDN connection. Inaddition, for a LAN connection, the processing is analogous as for theISDN connection, with the following differences:

The commstub task replaces the comm task.

The ALINKOUT₋₋ HOST₋₋ TMSG message replaces the ALINKOUT₋₋ TMSG message.

The ALINKIN₋₋ HOST₋₋ TMSG message replaces the ALINKIN₋₋ TMSG message.

The commstub task sends buffers to and receives buffers from the host.

Global Data Package

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

NULLDEV Driver

The SPOX image for the audio/comm board contains a device driver thatsupports interprocess communication though the stream (SS) package. Thenumber of distinct streams supported by NULLDEV is controlled by adefined constant NBRNULLDEVS in NULLDEV.H. NULLDEV supports threestreams. One is used for the audio task capture thread to communicatewith the comm task for ISDN connection. Another is used by the playbackthread to communicate with the comm task. The third is for the audiocapture task to communicate with the commstub task for LAN connection.The assignment of device names to tasks is done by the following threeconstants in ASTASK.H:

#define AS₋₋ CAPTURE₋₋ PIPE "/null"

#define AS₋₋ PLAYBACK₋₋ PIPE "/null2"

#define AS₋₋ HOST₋₋ CAPTURE₋₋ PIPE "/null3"

Support for additional streams may be obtained by changing theNBRNULLDEVS constant and recompiling NULLDVR.C. The SPOX config file isalso adjusted by adding additional device name strings to this sectionas follows:

    ______________________________________                                                 driver NULLDEV.sub.-- driver {                                                  "/null":                                                                             devid = 0;                                                             "/null2":                                                                            devid = 1;                                                             "/null3":                                                                            devid = 2;                                                           };                                                                   ______________________________________                                    

The next device in the sequence has devid=3.

SS₋₋ get() calls to NULLDEV receive an error if NULLDEV's ready queue isempty. It is possible to SS₋₋ put() to a NULLDEV stream that has notbeen opened for SS₋₋ get() on the other end. Data written to the streamin this case is discarded. In other words, input live buffers are simplyappended to the free queue. SS₋₋ put() never returns an error to thecaller. If no buffers exist on the free queue for exchange with theincoming live buffer, NULLDEV removes the buffer at the head of theready queue and returns it as the free buffer.

PWave Subsystem

The PWave subsystem provides high-priority playback of digital audiosignals contained in Microsoft® standard Wave files.

PWave API

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

High Priority Playback Task

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

PWave Protocol

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Comm Subsystem

The communications (comm) subsystem of conferencing system 100 of FIG. 5comprises:

Comm API 510, comm manager 518, DSP interface 528, and portions of thenetwork stacks 560 running on host processor 202 of FIG. 2,

Portions of the network stacks 560 running on LAN board 210, and

Comm task 540 running on audio/comm (ISDN) board 206.

The comm subsystem provides connectivity functions to the conferencingapplication programs 502 and 504. It maintains and manages the session,connection, and the virtual channel states. All the connection control,as well as data communication are done through the communicationsubsystem.

Referring now to FIG. 17, there is shown a block diagram of the commsubsystem of conferencing system 100 of FIG. 5. The comm subsystemconsists of the following layers that reside on host processor 202, theaudio/comm (ISDN) board 206, and LAN board 210:

Transport independent interface 510 (TII.DLL),

Datalink module 1702 (DLM.DLL+KPDAPI.DLL, where KPDAPI.DLL is theback-end of the DLM which communicates with the DSP interface 528),

Reliable datalink module 1704 (RDLM.DLL),

Global dynamic loader 1706 (GDL.DLL),

Global dynamic loader executable 1708 (GDLE.EXE),

Control (D channel) 1710,

D channel driver 1712,

Data comm tasks 1714,

B channel drivers 1716,

LAN datalink module 1718 (DLMLAN.DLL),

The appropriate LAN media dependent modules 1720 (MDM.DLLs),

The appropriate comm stacks 560, and

The MDM helper task 1722 (MDMHELPR.DLL).

TII 510, DLM 1702, DSP interface 528, RDLM 1704, DLMLAN 1718, the MDMs1720, portions of the comm stacks 560, MDMHELPR 1722, GDL 1706, andGDLE.EXE 1708 reside entirely on the host processor. Control (D channel)1710, D channel driver 1712, data comm tasks 1714, and B channel drivers1716 reside on audio/comm (ISDN) board 206. Portions of the comm stacks560 reside on the LAN board 210.

The comm interface provides a "transport independent interface" for theconferencing applications. This means that the comm interface hides allthe network dependent features of the conferencing system. For ISDNconnections, conferencing system 100 uses the ISDN Basic Rate Interface(BRI) which provides 2*64 KBits/sec data (B) channels and one signaling(D) channel (2B+D). Conferencing system 100 also uses conventional LANconnections.

The comm subsystem provides an interface by which the conferencingapplications can gain access to the communication hardware. The goal ofthe interface is to hide the implementation of the connectivitymechanism and provide an easy to use interface. This interface providesa very simple (yet functional) set of connection control features, aswell as data communication features. The conferencing applications usevirtual channels for data communication. Virtual channels are simplex,which means that two virtual channels are open for full duplexcommunication between peers. Each conferencing application opens itsoutgoing channel which is write-only. The incoming (read-only) channelsare created by "accepting" an "open channel" request from the peer.

ISDN-Based Conferencing

Referring now to FIG. 18, there is shown a block diagram of the commsubsystem architecture for two conferencing systems 100 participating ina conferencing session over an ISDN connection. The comm subsystemprovides an asynchronous interface between the audio/comm (ISDN) board206 and the conferencing applications 502 and 504.

The comm subsystem provides all the software modules that manage the twoISDN B channels. The comm subsystem provides a multiple virtual channelinterface for the B channels. Each virtual channel is associated withtransmission priority. The data queued for the higher priority channelsare transmitted before the data in the lower priority queues. Thevirtual channels are unidirectional. The conferencing applications openwrite-only channels. The conferencing applications acquire read-onlychannels as a result of accepting a open channel request from the peer.The DLM supports the virtual channel interface.

During an ISDN-based conferencing session, the comm subsystem softwarehandles all the multiplexing and inverse multiplexing of virtualchannels over the B channels. The number of available B channels (andthe fact that there is more than one physical channel available) is nota concern to the application.

The comm subsystem provides the D channel signaling software to theaudio/comm (ISDN) board. The comm subsystem is responsible for providingthe ISDN B channel device drivers for the audio/comm (ISDN) board. Thecomm subsystem provides the ISDN D channel device drivers for theaudio/comm (ISDN) board. The comm software is certifiable in NorthAmerica (U.S.A., Canada) and Europe. The signaling software iscompatible with NII, AT&T Custom, and Northern Telecom DMS-100.

LAN-Based Conferencing

For LAN-based conferencing, the comm subsystem provides an asynchronousinterface between the LAN board 210 and the conferencing applications502 and 504. The comm subsystem provides all the software modules thatmanage the LAN communication network 110. The comm subystem provides amultiple virtual channel interface for the LAN interconnecton betweenthe conferencing machines. Each virtual channel is associated with atransmission priority. The data queued for the higher priority channelsare transmitted before the data in the lower priority queues. Thevirtual channels are unidirectional. The conferencing applications openwrite-only channels. The conferencing applications acquire read-onlychannels as a result of accepting an open channel request from the peer.The DLMLAN modules supports the virtual channel interface.

During a LAN-based conferencing session, the comm subsystem handles allthe multiplexing and inverse multiplexing of virtual channels over thetypically singular LAN interconnection. The number of network `sockets`or connection points is not a concern to the application.

When the video conferencing connection is across the LAN, comm stack 506receives the compress audio generated by the remote site and stores itto host memory. The appropriate LAN MDM 1720 of FIG. 17 and DLMLAN 1718then reconstructs the compressed audio stream as the sequence of packetssupplied by the audio manager on the remote site to that site's LAN commsubsystem. The comm manager 518 then passes the audio packets to theaudio manager 520, which sends the packets to the audio task onaudio/comm (ISDN) board 206 for playback.

qMUX MULTIPLE CHANNEL STREAMING MODULE

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954. In addition, for LAN-based conferencing,the LAN implementation of the DLM interface (DLMLAN) 1718 provides thesame functionality on the LAN that DLM 1702 does for ISDN-basedconferencing, i.e., virtual channels and transport independent messagesizes. The DLMLAN implementation is supported on another abstractionlayer, the media dependent modules (MDMs) 1720. The MDMs have a commonMDM API and they implement the required functionality on top of anexisting LAN protocol stack (e.g., IPX, TCP/IP) A single MDM helper task(MDMHELPR) 1722 assists the MDMs by generating threads of execution forcallbacks and data transmission.

Comm API

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954. In addition, sessions and connections haveassociated addresses, represented by the TADDR structure. A TADDRconsists of a transport type and up to 80 bytes of addressinginformation. The transport type specifies if this is an ISDN or LANaddress. Referring again to FIG. 17, TII 510 determines which DLM willbe servicing a given address by passing it to the Global Dynamic Loader(GDL) module 1706. GDL 1706 and its associated helper task GDLE 1708load the appropriate module into memory and return all of the DLM entrypoints to TII 510. If this is a LAN address, the DLMLAN 1718 will thenconsult GDL 1706 in order to load the appropriate MDM 1720. DLMLAN 1718receives back from GDL 1706 a list of the appropriate MDM entry points.GDL 1706 and GDLE 1708 determine the appropriate DLM and MDM to load byreading the file GDL.INI which is written when the product is installed.This file specifies the MDMs that are appropriate based on theconfiguration of the user's machine. Further description of theoperations of global dynamic loaders and global dynamic loaderexecutables is presented in U.S. patent application Ser. No. 08/133,612,now U.S. Pat. No. 5,410,698. Additional information on the comm API isfound in APPENDIX E of this specification.

Automatic Transport Detection

Conferencing system 100 of FIG. 1 is capable of supporting conferencingover different types of transports (e.g., ISDN and LAN). Moreover,conferencing system 100 is capable of supporting LAN-based conferencingunder different LAN transport standards (e.g., Novell IPX, Internet UserDatagram Protocol (UDP), and/or NetBIOS standards). Further still,conferencing system 100 is capable of supporting LAN-based conferencingwith different LAN products for a single LAN transport standard (e.g.,LAN WorkPlace (LWPUDP) by Novell and FTPUDP by FTP Software, Inc., bothof which conform to the LAN UDP standard).

In order for a particular conferencing system 100 to be able to exercisethe full range of its conferencing options, it knows which of thesupported transports are installed. Conferencing system 100 is able todetermine automatically which supported transports are installed. Thisautomatic transport detection may be implemented at install time (i.e.,when conferencing system 100 is installed in a PC node) and/or at runtime (i.e., when conferencing system 100 is ready to beginconferencing).

Although different LAN products that conform to the same transportstandard will generate data with the same packet format, they may havedifferent APIs for generating and interpreting those packets. Thus,automatic transport detection determines which transport products areinstalled as well as which transport types and standards are supported.Each different supported transport will typically have a correspondingmedia dependent module (MDM). A goal of automatic transport detection isto identify (and store in the GDL.INI file) the specific MDMs to be usedto communicate with the specific network transport stacks that aresupported and installed in conferencing system 100.

Install-Time Processing

Conferencing systems 100 may be configured to support conferencing overdifferent sets of transports. For example, a particular conferencingsystem 100 may support conferencing over ISDN, Novell IPX, and UDP, butnot NetBIOS. The supported transports are presented to the conferencingapplication 502 of conferencing system 100 as a list of numberscorresponding to the supported transports. Possible supported transportsare identified as follows:

ISDN: 0

NetBIOS: 1

Novell IPX: 7

UDP: 8

For the conferencing system 100 in the example, a list of supportedtransports is (0, 7, 8).

Referring now to FIG. 42, there is shown a flow diagram of theprocessing by conferencing system 100 of FIGS. 5 and 17 during theautomatic transport detection implemented at install time. Theconferencing application selects the next (in this case, first)supported transport in the list of supported transports for thatparticular conferencing system 100 (step 4202 of FIG. 42).

Conferencing system 100 should have one or more possible correspondingMDMs for each supported transport, where there may be more than one MDMwhen there is more than one product for a particular transporttype/standard. The conferencing application selects and loads the next(in this case, first) possible MDM for the currently selected transport(step 4204). The conferencing application calls the MDM₋₋ BeginSessionfunction to attempt to initialize the network transport stack (step4206). A session may be defined as the use by an application of aspecific transport to send and/or receive data packets on a particularnetwork address. The conferencing application calls the MDM₋₋BeginSession function to request that a session be initiated by thelocal network stack.

If the attempt to begin a session is successful (step 4208), then thecurrently selected MDM is the MDM that corresponds to the networktransport stack (for the currently selected transport) that is actuallyconfigured in conferencing system 100. In that case, the identity of thecurrently selected MDM is written to the GDL.INI file for the currentlyselected transport (step 4210). Processing then continues to step 4214.

If, however, the attempt to begin the session is unsuccessful (step4208), then the currently selected MDM is not the correct MDM for theconfigured network transport stack. In that case, the conferencingapplication determines whether there are any other possible MDMs for thecurrently selected transport (step 4212). If so, then processing returnsto step 4204 to select and load the next possible MDM and attempt tobegin a session using it. If there are no more possible MDMs for thecurrently selected transport, then the conferencing applicationdetermines whether there are any more transports in the list ofsupported transports (step 4214). If so, then processing returns to step4202 to repeat the processing for the next supported transport.Otherwise, install-time automatic transport detection is complete.

A result of automatic transport detection is a GDL.INI file that has,for each configured transport, the correct MDM to service thattransport. The GDL.INI file is used by the conferencing application atrun time to select the MDM to load and use for conferencing over aparticular transport.

Run-Time Processing

Automatic transport detection is implemented at run time to determinewhich transports can be used for an impending conferencing session.Inputs to run-time automatic transport detection are the list ofsupported transports and the GDL.INI file that was generated by runningautomatic transport detection at install time. For each supportedtransport and using the corresponding MDM identified in the GDL.INIfile, the conferencing application attempts to begin a session. If theattempt to begin the session is successful, then the conferencingapplication knows it can use that transport for the conferencingsession.

Referring now to FIG. 43, there is shown a block diagram showing thenetwork connections made by conferencing system 100 of FIGS. 5 and 17during the automatic transport detection implemented at run time. Thesequence shown in FIG. 43 may be enumerated as follows:

(A) Conferencing application 502 calls conferencing manager 544 to begina session using a specified transport.

(B) Conferencing manager 544 passes begin session request toconferencing API 506.

(C) Conferencing API 506 passes begin session request to comm API 510.

(D) Comm API 510 calls GDLE.EXE 1708 to load the LAN data link manager(DLMLAN) 1718 corresponding to the specified transport.

(E) GDLE.EXE 1708 accesses the GDL.INI file 4302 to determine the filename for the DLMLAN 1718 that services the specified transport.

(F) GDLE.EXE 1708 loads the appropriate DLMLAN 1718 into memory andsends the corresponding entry points into DLMLAN 1718 back to comm API510.

(G) Comm API 510 calls DLMLAN 1718 to begin a session.

(H) DLMLAN 1718 calls GDLE.EXE 1708 to load the media dependent module(MDM) corresponding to the specified transport.

(I) GDLE.EXE 1708 accesses the GDL.INI file 4302 to determine the filename for the MDM that services the specified transport.

(J) GDLE.EXE 1708 loads the appropriate MDM 1720 into memory and sendsthe corresponding entry points into MDM 1720 back to DLMLAN 1718.

(K) DLMLAN 1718 calls MDM 1720 to begin a session.

(L) MDM 1720 tries to communicate with the network stack for which it isdesigned to begin a session.

If the MDM's attempt to communicate with the network stack issuccessful, then that success is communicated from MDM 1720 to DLMLAN1718 to comm API 510 to conferencing API 506 to conference manager 544to the conferencing application 502. The conferencing application 502then knows that it can use that transport for the impending conferencingsession. Similarly, if the MDM's attempt to communicate with the networkstack does not succeed, then that failure is communicated through thevarious levels to the conferencing application 502, which then knowsthat the conferencing session cannot proceed over that transport. Inthis latter case, as the MDM's attempt to begin the session fails,DLMLAN 1718 calls GDLE.EXE 1708 to unload MDM 1720, and, as the DLMLAN'sattempt to begin the session fails, comm API 510 calls GDLE.EXE 1708 tounload DLMLAN 1718.

The scenario presented in FIG. 43 is repeated for each of the supportedtransports in the conferencing application's list of supportedtransports to determine all of the transports that are currentlyavailable for the conferencing session. When presenting the user with alist of possible callee addresses for the conferencing session (as partof a dialog box), the conferencing application 502 only lists addressesthat correspond to transports that the conferencing application 502 hasdetermined are available.

Transport-Independent Interface

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned. These functions are defined infurther detail later in this specification in Appendix E.

Message and Callback Parameters

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now abandoned.

Session Handler Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Channel Manager Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Channel Handler Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

iTone

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned. Further description of the iTonestring and its use may be found in U.S. patent application Ser. No.08/305,206, filed Sep. 13, 1994, now U.S. Pat. No. 5,600,684.

Data Structures

Referring now to FIG. 19, there is shown a representation of the commsubsystem application finite state machine (FSM) for a conferencingsession between a local conferencing system (i.e., local site or caller)and a remote conferencing system (i.e., remote site or callee).Referring now to FIG. 20, there is shown a representation of the commsubsystem connection FSM for a conferencing

session between a local site and a remote site. Referring now to FIG.21, there is shown a representation of the comm subsystem controlchannel handshake FSM for a conferencing session between a local siteand a remote site. Referring now to FIG. 22, there is shown arepresentation of the comm subsystem channel establishment FSM for aconferencing session between a local site and a remote site. Referringnow to FIG. 23, there is shown a representation of the comm systemprocessing for a typical conferencing session between a caller and acallee. The description for this section is the same as the descriptionfor the section of the same name in U.S. patent application Ser. No.08/157,694.

Comm Manager

The comm manager 518 of FIG. 5 comprises the following dynamicallylinked libraries of FIG. 17:

Transport independent interface (TII) 510,

Reliable datalink module (RDLM.DLL) 1704,

Datalink module interface (DLM.DLL) 1702,

LAN datalink module interface (DLMLAN.DLL) 1718,

One or more media dependent modules (MDM.DLL) 1720,

Global dynamic loader (GDL.DLL) 1706,

Global dynamic loader executable (GDLE.EXE) 1708, and

MDM helper (MDMHELPR.DLL) 1722.

The DLM interface is used by the TII to access the services of the ISDNaudio/comm board 206. The DLMLAN interface is used by the TII to accessthe services of the LAN board 210. Other modules (i.e., KPDAPI.DLL andDSP.DRV) function as the interface to the audio/comm board and have noother function (i.e., they provide means of communication between thehost processor portion of the DLM and the audio/comm portion of the DLM.The host processor portion of the DLM (i.e., DLM.DLL) uses the DSPinterface 528 of FIG. 5 (under Microsoft® Windows™ 3.x operating system)to communicate with the ISDN audio/comm board side portions.

The TII provides the ability to specify whether or not a virtual channelis reliable. For reliable channels, TII employs the RDLM to providereliablility on a virtual channel. This feature is used to indicate thatthe audio and video virtual channels are unreliable, and the datavirtual channel is reliable.

Referring again to FIG. 17, TII 510 is a dynamic link library (DLL) thatimplements the comm API. There is only a single instance of the TIIlibrary running on the host and it supports multiple transport media andmultiple connections. At the bottom, the TII library makes DLM callsthat are directed to the specific DLM capable of handling the address(transport) type in question.

A data link manager (e.g., DLM 1702, DLMLAN 1718) handles one or moretransport types. A DLM provides:

Fragmentation and re-assembly of large messages,

Implementation of logical channels within a connection,

Prioritization of data across channels, and

In-order delivery of messages, with message boundaries preserved.

A DLM may directly interface to the transport media device (e.g., in thecase of ISDN connections) or it may call the relevant media dependentmodule (MDM) (e.g., in the case of LAN connections) for services.

All transport media specific functionality is encapsulated into a mediadependent module (MDM) 1720. There is one MDM per transportmedium/protocol. Possible MDMs are NetBIOS, IPX, POTS Modems, and TAPI(Mikado PBX). If the underlying transport medium inherently supportsmultiple connections (e.g., NetBIOS), then the MDM should provide it tothe upper layers. Some MDMs will provide a only single connection (e.g.,a POTS Modem MDM that supports a single external modem). The MDMprovides functionality for connection establishment and tear-down, andreliable data transfer over the connection(s). It does not have anyknowledge of logical data channels. In conferencing system 100, each MDMis implemented as a DLL.

Additionally, there are two support modules. The link packet manager(LPM) 1724 creates, destroys, and allocates link packets for thecommunications stack. A link packet is data structure shared between thehost-resident DLM and an MDM. Link packets allow for efficient transferof data between the DLM and MDM. The global dynamic loader (GDL) 1706 isresponsible for bringing DLMs and MDMs into the system as needed and fordiscarding them when they are no longer used.

Data Link Manager

Referring now to FIG. 29, there are shown diagrams indicating typicalTII-DLM connection setup and teardown sequences. The description forthis section is the same as the description for the section of the samename in U.S. patent application Ser. No. 08/340,172, filed Nov. 15,1994, now abandoned. One difference is the event structure EVENTSTRUCT,which is extended to return the data block on CONN₋₋ REQUESTED and isdefined as follows:

    ______________________________________                                                  EVENTSTRUCT                                                                struct EVENTSTRUCT                                                            {                                                                                WORD     EventType;                                                           WORD     Status;                                                              BYTE     DlmId;                                                               BYTE     MdmId;                                                               DWORD    DlmSessionId;                                                        DWORD    DlmConnId;                                                           DWORD    CallReference;                                                       DWORD    Token;                                                               LPTADDR  Addr;                                                                LPCONNCHR                                                                              Characteristics;                                                     LPVOID   UserData;                                                            WORD     UserDataLen;                                                      }                                                                      ______________________________________                                    

Parameters:

EventType Specifies the type of event which triggered the callback.

Status Indicates the status of the event.

DlmId Unique ID of the DLM performing the callback.

MdmId Unique ID of the MDM which processed the event.

DlmSessionId Indicates the Session ID, assigned by DLM, on which thisevent occurred.

DlmConnId Indicates the Connection Id, assigned by DLM, on which thisevent occurred.

Token The token value was given in the call to initiate an action. Whenthe callback notifies the user that the action is complete, the token isreturned in this field.

Addr Specifies the LPTADDR of the caller.

Characteristics This field is a LPCONNCHR to the connectioncharacteristics.

UserData Pointer to the data specified in the UserData parameter of theDLM₋₋ MakeConnection call for this connection.

UserDataLen Number of valid bytes in the UserData block.

The UserData and UserDataLen fields are valid only for the CONN₋₋REQESTED callback and may be used for CallerID information. If the userdid not specify any data, the UserData field is NULL and UserDataLen is0.

Other differences are the DLM₋₋ MakeConnection and DLM₋₋RejectConnection functions, which are defined as follows:

DLM MakeConnection

WORD DLM₋₋ MakeConnection (DWORD DlmSessionId, LPCONNCHRCharacteristics, DWORD Token, LPTADDR RemoteAddress, LPDWORD DLMConnID,LPVOID UserData, WORD UserDataLen);

Parameters:

DlmSessionID identifier returned in DLM₋₋ BeginSession

Characteristics Desired characteristics of the connection. Passeduninterpreted to the lower layers.

Token Uninterpreted token returned to the upper layer in the responsecallback.

RemoteAddress Address on the remote site on which to make theconnection.

DLMConnID Output parameter specifying the DLM connection ID that will bevalid when this connection is established.

UserData Pointer to up to 64 bytes of user defined data that is to betransmitted to the remote site with the connection request.

UserDataLen Number of bytes in the UserData block. If more than 64 bytesare specified, the first 64 are transmitted.

DLM RejectConnection

WORD DLM₋₋ RejectConnection (DWORD DlmConnId, DWORD CallReference, WORDReasonCode);

Parameters:

DlmConnID Connection identifier returned in the CONN₋₋ REQESTEDcallback.

CallReference Identifier returned previously in the CONN₋₋ REQESTEDcallback.

ReasonCode Uninterpreted word that is transmitted to the remote site asthe reason for the rejection.

The reason code will be returned in the Status field of the DLM eventstructure for the CONN₋₋ REJECTED callback. If the remote user did notspecify a reason, the Status will be 0.

Interfaces--Channel Management & Data Transfer

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

MDM Interface

The media dependent module (MDM) 1720 of FIG. 17 hides the networkspecifics from the layers above it on the communications stack. It isthe only module that is affected by a change in the physical network.Media dependent modules are described in further detail in U.S. patentapplication Ser. No. 08/133,612, now U.S. Pat. No. 5,410,698. Additionalinformation on the MDM API is found in APPENDIX F of this specification.

MDMHelpr

The MDM helper component (MDMHelpr) 1722 of FIG. 17 is a server of allsystem level services to its client MDMs. In order to centralizecritical system timer resources, MDMHelpr 1722 task centralizes allaccess Microsoft® Windows™ messages, task time timers, multi-mediatimers and MDM messages. The first MDM starts the MDMHelpr task, whichcreates a ten-millisecond multimedia timer, a one-second task timetimer, and a client registration service. Subsequently, each MDMregisters as a client for the timers and message passing services. Thiscentralizes the Microsoft® Windows™ system resources which aremultiplexed within the helper eliminating redundancy as new MDMs areloaded. The MDMs utilize the helper's multimedia timer for both send andreceive queue processing, the one-second task timer for packet retriesand updating the log file, and the message queues for communicationmessages and control flow. As each MDM is closed, the clients areregistered with the last MDM causing the shutdown of the helper task.Also, during abnormal termination, the helper catches the Microsoft®Windows™ close message and correctly cleans up its resources avoidingcrashes by Microsoft® Windows™ from improper shutdown. Additionalinformation on the MDMHelpr API is found in APPENDIX G of thisspecification.

Link Packet Manager

The link packet manager (LPM) 1724 of FIG. 17 maintains the pool of freelink packets. Both DLM and MDM request link packets and, when they arefinished with them, send them back to the LPM to be returned to the freepool. Since requests for link packets can occur at interrupt time, theLPM can not allocate packets on the fly. It allocates all of its freepool of packets when it is initialized and continues to re-use onlythose packets. Therefore, both DLM and MDM are able to handle the casethat a link packet is not available. Link packet managers are describedin further detail in U.S. patent application Ser. No. 08/133,612, nowU.S. Pat. No. 5,410,698. Additional information on the LPM API is foundin APPENDIX H of this specification.

Global Dynamic Loader

The global dynamic loader (GDL) 1706 of FIG. 17 is responsible forloading all necessary DLMs and MDMs into memory. The advantage overstatically loading the libraries is that the communications stack neednot be fixed when the application is started. The application may decidewhich network transport to use and consequently which MDM to load.Global dynamic loaders are described in further detail in U.S. patentapplication Ser. No. 08/133,612, now U.S. Pat. No. 5,410,698. Additionalinformation on the GDL API is found in APPENDIX I of this specification.

DSP Interface

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Comm Task

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

LAN Management Interface (LMI) Subsystem

The LAN management interface (LMI) module controls the communicationsbetween conferencing system 100 and a management computer to obtain LANbandwidth allocations from the management computer for conferences. LMIconsists of LMI API 556 and LMI manager 558 of FIG. 5. LMI maintains aninternal interface to its own windows application (LMITRD.EXE) whichprovides a windows task thread to LMI. A management computer manages theallocation of bandwidth for conferences on a network. Managementcomputers and how they manage bandwidth allocation for networkconferences are described in greater detail in U.S. patent applicationSer. No. 08/340,172, filed Nov. 16, 1994, now abandoned, entitled"Managing Bandwidth Over a Computer Network" of Robert AlexanderMarshall, et al. Additional information on the LMI API is found inAPPENDIX J of this specification.

Application-Level Protocols

The application-level protocols for conferencing system 100 of FIG. 5are divided into those for the video, audio, and data streams.

Video Protocol

Referring now to FIG. 24, there is shown a representation of thestructure of a video packet as sent to or received from the commsubsystem. The description for this section is the same as thedescription for the section of the same name in U.S. patent applicationSer. No. 08/340,172, filed Nov. 15, 1994, now abandoned. In addition,conferencing system 100 is capable of encoding and decoding videosignals in more than one bitstream format. Conferencing system 100supports an ISDN rate video (IRV) bitstream format and a multi-ratevideo (MRV) bitstream format. These formats are described in thefollowing sections.

Compressed ISDN Rate Video Bitstream

Referring now to FIG. 25, there is shown a representation of thecompressed ISDN rate video (IRV) bitstream for conferencing system 100.The description for this section is the same as the description for thesection entitled "Compressed Video Bitstream" in U.S. patent applicationSer. No. 08/157,694, now U.S. Pat. No. 5,506,954.

Decoding Procedure for IRV Bitstream Format

The description for this section is the same as the description for thesection entitled "Video Decoding Procedure" in U.S. patent applicationSer. No. 08/157,694, now U.S. Pat. No. 5,506,954.

Intra/Inter Decision Rules

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Post Reconstruction Loop Filtering

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Adaptive Loop Filter Switching Criteria

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Design of Quantization Tables

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Adaptive Transform Coefficient Scanning

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Spatially Adaptive Quantization

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Fast Statistical Decode

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Contrast, Brightness and Saturation Controls

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Compressed Multi-Rate Video Bitstream

Encode and decode processing of one embodiment of the multi-rate video(MRV) bitstream are described in U.S. patent application Ser. No.08/235,955, now U.S. Pat. No. 5,493,514.

Picture Layer

According to a current embodiment, the compressed data for one picturehas the following format:

    PictureHeader  SliceData!  SliceHeader SliceData!  SliceHeader SliceData! . . .

The format of the picture header is as follows:

    ______________________________________                                        # Bits       Field                                                            ______________________________________                                         *    18         Picture start code (00000000 00000000 10)                          2          VersionNumber                                                      7          FrameNumber                                                        5          SliceStructure                                                *    3          PictureSize                                                        4          Reserved                                                           1          ExtraInfo                                                     *    3          QuantMatrices                                                      2          TempFiltStrength                                                   1          UsesSLF                                                            1          UsesBlockMV                                                        1          IntraFlag                                                     *    8          PictureXsize!                                                 *    8          PictureYsize!                                                 *    16         ExtraInfoSize!                                                *    var        ExtraInfoData!                                                *    var        QuantData!                                                   ______________________________________                                    

where * indicates a field that starts on a byte boundary, ! indicates anoptional field, and var indicates a variable number of bits. The meaningof these fields is as follows:

Picture start code A unique code that cannot occur anywhere else in thebit stream that identifies the start of a frame.

VersionNumber Identifies a version of the MRV video bit stream. Thecurrent version number is 1.

FrameNumber A counter that indicates to the decoder the receipt ofsuccessive frames. An encoder should increment this field by 1 for eachframe it encodes (with wraparound, so that the FrameNumber after 127 is0). A decoder can then determine by looking at this field if a frame hasbeen "lost," except for the rare case in which a multiple of 128 framesin a row are lost.

SliceStructure Optionally specifies the slice structure of the picture.The binary value of this field has the following meanings:

    ______________________________________                                        SliceStructure                                                                         Meaning                                                              ______________________________________                                        0        No slice structure specified here. Each slice has a slice                     header.                                                              1        All slices are of size 1, planes are encoded in order YUV.           2        All slices are of size 1, planes are encoded in order UVY.           3        Each plane is encoded as a slice, planes are encoded                          in order YUV.                                                        4        Each plane is encoded as a slice, planes are encoded                          in order UVY.                                                        5-31     Reserved.                                                            ______________________________________                                    

If SliceStructure=0, information about slices is contained in the sliceheaders which appear later in the bitstream. If SliceStructure=1, 2, 3,or 4, the slice structure of the picture is specified here, as shown inthe table above, and there are no slice headers in the bitstream forthis picture.

PictureSize Identifies the size of the encoded picture. The binary valueof this field has the following interpretation:

    ______________________________________                                        PictureSize                                                                            Resolution                                                           ______________________________________                                        0         80 × 60                                                       1        160 × 120                                                      2        240 × 180                                                      3        320 × 240                                                      4-6      reserved                                                             7        escape - size is in PictureXsize and PictureYsize                    ______________________________________                                                 fields.                                                          

Reserved This field is reserved for future use, and is set to zero.

ExtraInfo This bit specifies whether there is extra information in thepicture header.

QuantMatrices Specifies what quantization matrices the decoder shoulduse for this frame. There are 32 quantization matrices to be defined: 16for use in intra blocks and 16 for use in inter blocks. These 32matrices are specified by the contents of two base matrices (one forintra, one for inter), five quantization parameters, and a flag(PowersOf2). The possible values of the QuantMatrices field specify whatthe base matrices, parameters, and PowersOf2 flag are, according to thefollowing table:

    ______________________________________                                        QuantMatrices                                                                           BaseMatrices                                                                              Parameters PowersOf2                                    ______________________________________                                        0         ...not used...                                                      1         default     default    0                                            2         default     default    1                                            3         default     in QuantData                                                                             in QuantData                                 4         in QuantData                                                                              in QuantData                                                                             in QuantData                                 5         from the past                                                                             from the past                                                                            from the past                                6-7       ...reserved...                                                      ______________________________________                                    

In this table, "in QuantData" means that the given item is to be foundin the QuantData field of the picture header. "From the past" means thatthe parameter values (which were set on a previous frame) are inheritedfrom the past.

TempFiltStrength Specifies the strength of the temporal loop-filter. Ifthis field is 00, the temporal filter is turned off for this picture.

UsesSLF Specifies whether this picture uses the spatial loop filter ornot. The setting of this bit changes the meaning of macroblock typesread from the bit stream.

UsesBlockMV If set, specifies that this picture may contain block-based(as opposed to macroblock-based) motion vectors. The setting of this bitchanges the meaning of macroblock types read from the bit stream.

IntraFlag If set to 1, denotes that this picture is entirely intracoded.

PictureXsize and PictureYsize

Contain the picture sizes divided by 4. Since these are 8-bit fields,this means that MRV video can support picture sizes up to 1020×1020.These fields are present only if the PictureSize field=7.

ExtraInfoSize This field is present only if the Extralnfo bit is setearlier in the picture header. It specifies the number of bytes of extrainformation (not including this field) that is present in the pictureheader.

ExtraInfoData Extra information for private use by an encoder ordecoder. An MRV video decoder should simply skip over any ExtraInfoDatapresent in a bitstream.

QuantData If present (as indicated by the QuantMatrices field), containsa definition of the quantization parameters, the PowersOf2 flag, andpossibly the two base matrices as well. If this field is present, itsfirst four bytes are as follows:

    ______________________________________                                        # Bits             Field                                                      ______________________________________                                        6                  QuantStart                                                 6                  QuantStep                                                  6                  DCstep                                                     6                  Tilt 0! (for inter)                                        6                  Tilt 1! (for intra)                                        1                  PowersOf2                                                  1                  Reserved                                                   ______________________________________                                    

If indicated by the QuantMatrices field, the definition of the two basematrices follows. Each matrix consists of 64 6-bit fields packed into 48bytes. The inter base matrix is first, followed by the intra. Eachmatrix is stored in "raster-scan" order.

Slice Layer

The data for one MRV video picture consists of a series of data groupsfor one or more slices. A slice is a contiguous group of one or morerows of macroblocks in the picture. The slices present in the bit stream"covers" all three planes (Y, U, and V) of the picture exactly (i.e.,the whole picture is coded and no part of the picture is encoded morethan once). Some slices may contain very little data (for example, ifall macroblocks in the slice are empty), but they are all present in thedata stream. Each slice in a picture is specified by start and sizefields specifying the row and plane where the slice starts and its size(i.e., how many rows of macroblocks the slice contains). A slice isrequired to be confined to a single plane. The data for one sliceconsists of three sections, as follows:

    SliceHeader MacroblockData BlockData

The format of the slice header is as follows:

    ______________________________________                                        # Bits      Field                                                             ______________________________________                                        18          Slice start code (00000000 00000000 11)                           6           SliceStart                                                        6           SliceSize                                                         2           Reserved                                                          ______________________________________                                    

where:

Slice start code Identifies the start of a slice. This unique codecannot occur elsewhere in the bit stream.

SliceStart Specifies where the slice starts in the picture. The units ofSliceStart are interpreted as follows: Take the rows of macroblocks ineach of the Y, U, and V planes and arrange them in scan-line order, thenconcatenate the rows for the planes into one long list. The value ofSliceStart is the (zero-based) index into this list. For example, in a(160×120) picture, there are 8 rows of macroblocks in the Y plane and 2rows in each of the U and V planes. So SliceStart would take on a valueof 0 to 11, where 0 represents the top row of macroblocks in the Yplane, 7 is the bottom row of the Y plane, 8 is the top row of the Uplane, etc.

SliceSize Specifies the size of the slice in rows of macroblocks. Aslice is confined to a single plane, but is allowed to start and end onany row. A value of SliceSize which describes a slice extending past theend of the plane is illegal. Slices in a picture need not appear in thebit stream "in order". For example, a picture could have 4 slices, inthe following order: U plane, top half of Y plane, V plane, bottom halfof Y plane.

Reserved A 2-bit field reserved for future use (0 for now).

Following the slice header is the Huffman-encoded macroblock and blockdata for one slice. The macroblock data always starts on a byteboundary, but the block data need not, as it is simply concatenated tothe end of the macroblock data.

Macroblock Layer

The macroblock data describes the structure of each macroblock in aslice. The macroblock data consists of a series of "records" of theform:

    empty  empty . . . ! type  Qvalue!  MV!  MV2 MV3 MV4!  cbp!

followed by the "separator" symbol 00011111, which separates themacroblock data from the block data for the slice. All of the symbols inthe above record are encoded using the Macroblock Huffman table. Themeaning of each of these fields is as follows:

The empty field gives information about how many empty macroblocks thereare between this non-empty macroblock and the previous one. There may bemore than one empty value, to indicate a long run of empty macroblocks.

The type field actually contains several bits of information about themacroblock. The decoded Huffman value is used as an index into one ofthe following four tables. The table to use is determined (on aper-picture basis) by the settings of the UsesSLF and UsesBlockMV bitsin the picture header.

    ______________________________________                                        Table for UsesSLF = 0 and UsesBlockMV = 0:                                    Huff Value  Intra  NewQ        MV   Cbp                                       ______________________________________                                        0           0      0           0    1                                         1           0      0           1    1                                         2           0      1           0    1                                         3           0      1           1    1                                         4           1      0           0    0                                         5           1      1           0    0                                         6           0      0           1    0                                         ______________________________________                                    

    ______________________________________                                        Table for UsesSLF = 1 and UsesBlockMV = 0:                                    Huff Value                                                                              Intra    NewQ    MV     Cbp  SLF                                    ______________________________________                                        0         0        0       0      1    0                                      1         0        0       1      1    1                                      2         1        0       0      0    0                                      3         0        1       0      1    0                                      4         0        1       1      1    1                                      5         0        0       1      0    1                                      6         1        1       0      0    0                                      7         0        0       1      1    0                                      8         0        0       1      0    0                                      9         0        1       1      1    0                                      10        0        0       0      1    1                                      11        0        1       0      1    1                                      ______________________________________                                    

    ______________________________________                                        Table for UsesSLF = 0 and UsesBlockMV = 1:                                    Huff Value                                                                            Intra    NewQ    MV     Cbp  BlockMV                                  ______________________________________                                        0       0        0       0      1    0                                        1       0        0       1      1    0                                        2       0        1       0      1    0                                        3       0        1       1      1    0                                        4       1        0       0      0    0                                        5       1        1       0      0    0                                        6       0        0       1      0    0                                        7       0        0       1      1    1                                        8       0        1       1      1    1                                        9       0        0       1      0    1                                        ______________________________________                                    

    ______________________________________                                        Table for UsesSLF = 1 and UsesBlockMV = 1:                                    Huff Value                                                                            Intra   NewQ    MV    Cbp  SLF   BlockMV                              ______________________________________                                        0       0       0       0     1    0     0                                    1       0       0       1     1    1     0                                    2       1       0       0     0    0     0                                    3       0       1       0     1    0     0                                    4       0       1       1     1    1     0                                    5       0       0       1     0    1     0                                    6       1       1       0     0    0     0                                    7       0       0       1     1    0     0                                    8       0       0       1     0    0     0                                    9       0       1       1     1    0     0                                    10      0       0       1     1    1     1                                    11      0       1       1     1    1     1                                    12      0       0       1     0    1     1                                    13      0       0       1     1    0     1                                    14      0       0       1     0    0     1                                    15      0       1       1     1    0     1                                    16      0       0       0     1    1     0                                    17      0       1       0     1    1     0                                    ______________________________________                                    

The bits in these tables have the following meaning:

Intra Says whether this macroblock is intra or not.

NewQ Says whether a quantization index (Qvalue) is present in thisrecord.

MV Says whether a motion vector is present in this record.

Cbp Says whether a Cbp (coded block pattern) value is present in thisrecord.

SLF Says whether the spatial loop filter is to be used for thismacroblock.

BlockMV Says whether this record contains four motion vectors (one foreach block) or one.

Following the type field are the QValue, MV, and cbp fields, which arepresent only if indicated by the corresponding bit in the type.

QValue is the Huffman-encoded differential quantization value. Thequantization value for this macroblock (thisQ) is calculated from theprevious macroblock's value (prevQ) as follows:

    thisQ=prevQ+tosigned(Qvalue+1),

where tosigned() is a function which converts from an unsigned number toa signed number. The resulting thisQ value is a 4-bit value thatspecifies which of 16 quantization matrices to use. The value of prevQis initialized to 8 at the start of each slice.

If MV=1, there is either one (if BlockMV=0) or four (if BlockMV=1)motion vectors present in this record. Each motion vector consists oftwo separate values, one for the x component and one for the ycomponent. For both x and y, the actual vector component is calculatedas:

    thisMV=prevMV+tosigned(huffman.sub.-- decoded.sub.-- value)

    if(thisMV>21) thisMV-=43;

    if(thisMV<-21) thisMV+=43;

In these equations, prevMV is the motion vector of the previous block ormacroblock, depending on whether this macroblock has one or four motionvectors, and whether the previous macroblock had one or four motionvectors, as follows:

    ______________________________________                                        #MVs in:        Previous motion vector for:                                   prev   this     MV(or MV1) MV2    MV3  MV4                                    ______________________________________                                        1      1        MVprev                                                        1      4        MVprev     MV1    MV2  MV3                                    4      1        MV4prev                                                       4      4        MV4prev    MV1    MV2  MV3                                    ______________________________________                                    

MVn corresponds to block number n in the macroblock (according to thenumbering shown in Macroblock and Block Structure). At the start of eachrow of macroblocks, the x and y components of prevMV are reset to zero.prevMV refers to the immediately-preceding macroblock. In particular,this means that if the preceding macroblock is empty, prevMV=0. Apositive motion vector x component means that the prediction block inthe previous picture is to the right of the block in the currentpicture. A positive y component means that the prediction block is belowthe block in the current picture. The x and y components are values inthe range -21 to +21.

The cbp value in the record specifies which blocks in the macroblock areempty (i.e., have no coded transform coefficients). The cbp value isobtained by taking the Huffman-decoded value (which will be in the range0-14), and indexing into the following table:

    Cbp Lookup Table: 15, 9, 3, 14, 7, 1, 11, 6, 2, 8, 13, 4, 12, 10, 5

cbp is never zero, since in that case the Cbp bit in the type fieldwould be set to 0, and cbp would not be present in the record. The 4-bitcbp value specifies the emptiness of the 4 blocks in the macroblock,with bit=1 meaning "non-empty". The LSB of cbp applies to block #1, thenext bit to block #2, etc. There are also two special cases:

(1) If the macroblock type says "intra," cbp is not present but isimplied to be 15.

(2) If the macroblock type says "non-intra" and "cbp not present," cbpis implied to be 0.

Block Layer

The block data contains data for each of the coded (non-empty,non-phantom) blocks in a slice. Blocks are coded in macroblock scanorder, and within each macroblock in counterclockwise order starting atthe top left corner. The block data is terminated by a string of 14 zerobits, which is the first unused code in the block Huffman table. Thereare two cases:

(1) If the block data is followed by a slice header for another slice,the block data is padded (with zero bits) to the next byte boundary.This can be followed by a string of or more "padding bytes" of zeros,followed by the slice header for the next slice. Regardless of thenumber of padding bits or bytes, the zero bits which begin the nextslice header guarantee that the block data is terminated by a string of14 zero bits.

(2) If the block data is not followed by a slice header (either becausethe picture header was used to specify the slice structure, or becausethis is the last slice of the picture), then the encoder explicitlyinserts the 14 zero bits. Then, the block data is padded with zero bitsto the next byte boundary. If another (header-less) slice follows, itsmacroblock data follow immediately and padding bytes are not allowed.

In both cases, the macroblock data for a slice starts on a byteboundary. The general format of the block data is as follows:

     Block1data!  Block2Data! . . . 00000000000000

and the format of each block's data is as follows:

     RunVal code or ESC run valLO valHI! . . .  RunVal code or ESC run valLO valHI! . . . EOB

The basic unit of block data is the "run/value pair" or run/val pair forshort. Each run/val pair represents one non-zero FST frequency-domaincoefficient. The commonly-occurring run/val pairs are encoded with asingle code; others are encoded as ESC followed by the explicit run andvalue.

Audio Protocol

Referring now to FIG. 26, there is shown a representation of acompressed audio packet for conferencing system 100. The description forthis section is the same as the description for the section of the samename in U.S. patent application Ser. No. 08/340,172, filed Nov. 15,1994, now abandoned.

Compressed Audio Bitstream

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Data Protocol

The description for this section is the same as the description for thesection of the same name in U.S. Patent Application filed Nov. 15, 1994.Data conferencing application 504 is described in greater detail in U.S.patent application Ser. No. 08/137,319 (filed Oct. 14, 1993), now U.S.Pat. No. 5,452,299 and in U.S. patent application Ser. No. 08/170,146(filed Dec. 20, 1993), now U.S. Pat. No. 5,581,702.

Communication-Level Protocols

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694,now U.S. Pat. No. 5,506,954.

Reliable Transport Comm Protocols

Referring now to FIG. 27, there is shown a representation of thereliable transport comm packet structure. The description for thissection is the same as the description for the section of the same namein U.S. patent application Ser. No. 08/157,694, now U.S. Pat. No.5,506,954.

Unreliable Transport Comm Protocols

Referring now to FIG. 28, there is shown a representation of theunreliable transport comm packet structure. The description for thissection is the same as the description for the section of the same namein U.S. patent application Ser. No. 08/340,172, filed Nov. 15, 1994, nowabandoned.

DLMLAN Protocols

The DLMLAN 1718 of FIG. 17 fragments the messages from TII 510 intopackets to be transmitted to the network. As shown in FIG. 44, eachpacket contains the following header:

    ______________________________________                                        Byte Offset                                                                             Item                                                                ______________________________________                                        0         Size of packets in bytes. (Format: WORD)                            2         Remote receiving channel number. (Format: BYTE)                     3         Local originating channel number. (Format: BYTE)                    4         Offset into the current message of the start of                               this packet. (Format: WORD)                                         6         Total size of the current message in bytes.                                   (Format: WORD)                                                      ______________________________________                                    

Following the DLM header is an implementation defined number of databytes.

MDM Protocols

For LAN transmissions, the MDMs 1720 of FIG. 17 wrap the DLM packet withthe following header, as shown in FIG. 45:

    ______________________________________                                        Byte Offset                                                                           Item                                                                  ______________________________________                                        0       Signature on packet. (Format: 3 BYTEs containing ASCII                        representation of "MDM")                                              3       Function code. A value of 0 represents a data packet.                         (Format: BYTE)                                                        4       MDM identifier for target connection. (Format: WORD)                  6       Sequence number. Monotonically increasing for packets                         on a single connection. (Format: WORD)                                8       16-bit CRC checksum on data in packet. (Format: WORD)                 10      Reserved. (Format: WORD)                                              ______________________________________                                    

Following the header is an implementation defined number of bytescontaining the data of the packet.

MDM Protocol for Connection Setup and Tear Down

The MDMs place logical connections on top of typically connectionlessnetwork protocols. Therefore, the MDMs exchange messages which build theMDM logical connection. There are four messages exchanged by the MDMs:connection request, connection acknowledgment, connection close, andclear to send.

Connection Request

The connection request message (Open) requests the creation ofconnection between two MDMs. This message requests the target MDM torespond using its listening connection to begin the connection acceptand establish sequence. The response expected is either an ACK or NAKmessage. If neither is received in one second from transmitting theOpen, another Open is sent. If no response is received by the requesterwithin the timeout period, then the Open request is cancelled. Theconnection request is sent as the result of a call to MDM₋₋MakeConnection. It has the following format:

    ______________________________________                                        Byte Offset                                                                           Item                                                                  ______________________________________                                        0       Signature on packet. (Format: 3 BYTEs containing ASCII                        representation of "MDM")                                              3       Function code. A value of 1 represents a connection                           request. (Format: BYTE)                                               4       Reserved. (Format: 0)                                                 6       MDM identifier for source connection making request.                          (Format: WORD)                                                        8       Flags, indicates if checksumming should be done on the                        packets. Set if the value is 1. (Format: WORD)                        10      Source Address. (Format: Variable size field depending                        on transport. For IPX, format: WORD containing the                            source socket. For UDP, format: DWORD containing UDP                          address of source followed by WORD containing the                             source socket. For NetBIOS, format: 16 bytes                                  specifying the source name of the endpoint.                           ______________________________________                                    

Immediately following source address (byte offset: 12 for IPX, 16 forUDP, 26 for NetBIOS) is the size of the user data block following themessage (format: WORD). Immediately following the user data block size(byte offset: 14 for IPX, 18 for UDP, 28 for NetBIOS) is the user datablock (format: uninterpreted block of up to 64 bytes of data sent to theMDM in MDM₋₋ MakeConnection), which may be used for CallerIDinformation.

Connection Acknowledgment

The connection acknowledgment message (ACK) is the response to the Openrequest, if the receiver has a connection listening and expects toestablish a connection. The acknowledgement of the connection requesthas the following format:

    ______________________________________                                        Byte Offset                                                                           Item                                                                  ______________________________________                                        0       Signature on packet. (Format: 3 BYTEs containing ASCII                        representation of "MDM")                                              3       Function code. A value of 2 represents a connection                           acknowledgement. (Format: BYTE)                                       4       MDM identifier for target connection request completing.                      (Format: WORD).                                                       6       MDM identifier for source connection making request.                          (Format: WORD)                                                        8-      Reserved.                                                             ______________________________________                                    

Any remaining bytes of the packet are 0.

Connection Close

The connection close message (NAK) is the response to the connectionrequest, if the receiver has no connection listening or does not expectto establish a connection. NAK is the request by either side of anaccepted or established connection to close or destroy the connection.Currently, there is no response to this request. The close or rejectmessage has the following format:

    ______________________________________                                        Byte Offset                                                                           Item                                                                  ______________________________________                                         0      Signature on packet. (Format: 3 BYTEs containing ASCII                        representation of "MDM")                                               3      Function code. A value of 3 represents a close                                connection message. (Format: BYTE)                                     4      MDM identifier for target connection request completing.                      (Format: WORD)                                                         6      MDM identifier for source connection making request.                          (Format: WORD)                                                         8      Flags, contains the reject or close reason code.                              (Format: WORD)                                                        10 -... Reserved.                                                             ______________________________________                                    

Any remaining bytes of the packet are 0.

Clear To Send

The clear to send message (CTS) is the request by the receiver of theconnection request message that the connection has been accepted andthat the connection is ready to receive data messages. The responseexpected is either CTS, NAK, or a connection data message. If noresponse is received by the requestor in one second from transmittingthe CTS request, another CTS is sent. If no response is received by therequestor within the timeout period, the CTS request is cancelled andthe connection is closed. The response to the CTS request if thereceiver has requested a connection be opened and expects to establish aconnection. A clear to send message has the following format:

    ______________________________________                                        Byte Offset                                                                           Item                                                                  ______________________________________                                         0      Signature on packet. (Format: 3 BYTEs containing ASCII                        representation of "MDM")                                               3      Function code. A value of 4 represents a clear to                             send indication. (Format: BYTE)                                        4      MDM identifier for target connection.                                         (Format: WORD)                                                         6      MDM identifier for source connection.                                         (Format: WORD)                                                         8      Flags, 0 if this is a clear to send request, 1                                if this is a clear to send response. (Format: WORD)                   10-...  Reserved.                                                             ______________________________________                                    

Any remaining bytes of the packet are 0.

Referring now to FIG. 46, there is shown a representation of theconnection messages for a typical conferencing session from theperspective of the MDMs on the local and remote nodes.

Feature and Capability Negotiation

Conferencing Management

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Connection Management

Connection and Channel Setup

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Connection Shutdown and Error Handling

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Conference Login

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Capability Negotiation

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Capabilities Structure

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Requests and Responses

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CallerCapRequest

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CallerCapCancel

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CalleeCapAccept

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CalleeCapReject

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

CalleeCapResponse

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Video Negotiation

Conferencing system 100 is able to support different modes of videoprocessing, where each different mode is defined by a set of fourparameters:

(1) Bitstream format: the format of the compressed video bitstream (mayalso imply the techniques used to code and decode the compressed videostream).

(2) Frame rate: the maximum number of video frames per second.

(3) Frame resolutions: the dimensions of the video frames in number ofpixels wide and number of pixels high.

(4) Bit rate: the maximum bit rate for the compressed video bitstream.

In a possible embodiment of the present invention, these four parametersare independent and theoretically limitless in variety. In anotherembodiment, however, the parameters are both limited and dependent. Asto the limits to the parameters:

(1) The possible bitstream formats are limited to the so-called IRVbitstream (as defined in FIG. 25) and the so-called MRV bitstream (asdefined in this specification in the section entitled "CompressedMulti-Rate Video Bitstream").

(2) The possible maximum frame rates are limited to integer numbers offrames per second from 1 to 30 fps, inclusive.

(3) The possible frame resolutions are limited to (160×120) and(320×240).

(4) The possible maximum bit rates are limited to 200 kilobits persecond (kbps), 90 kbps, and 84 kbps.

As to the interdependency of the parameters, the allowable video modesare limited to a specified set of supported combinations of the fourparameters. The set of supported combinations are enumerated in a tablein the video subsystem, which identifies the relative preferences forthe combinations. For example, a possible table may contain thefollowing supported video mode:

    ______________________________________                                              Bit      Frame             Bit     32-bit                               Choice                                                                              Stream   Rate     Resolution                                                                             Rate    Code                                 ______________________________________                                        1st   MRV      15 fps   (160 × 120)                                                                      200 kbps                                                                              2048                                 2nd   MRV      10 fps   (160 × 120)                                                                       90 kbps                                                                              1024                                 3rd   MRV      10 fps   (160 × 120)                                                                       84 kbps                                                                              512                                  4th   IRV      10 fps   (160 × 120)                                                                       84 kbps                                                                              2                                    ______________________________________                                    

Two nodes that want to have a video conference will negotiate with eachother to select a video processing mode (i.e., a specific combination ofthe four parameters) for the conferencing session. In one mechanism forconducting video negotiation, each step in the video negotiationinvolves one node sending to the other node a capabilities datastructure that encodes a particular set of video capabilities The datastructure contains a 32-bit field encoding the capabilities in separatebits, where each bit is a yes/no value for a different combination suchas those listed in the previous table. For example, the capabilities forall four video mode choices listed in the previous table may be encodedas follows:

    2 ORed with 512 ORed with 1024 ORed with 2048=0x00000E02

In an alternative embodiment of the present invention, the mechanism forconducting video negotiation between two conferencing systems 100 ofFIG. 1 assumes that the four video parameters are in fact independent.Under this mechanism, each step in the video negotiation involves onenode sending to the other node a capabilities data structure thatencodes a particular set of video capabilities. The data structurecontains a 32-bit field, such that:

One byte encodes bitstream formats, where each bit is a yes/no value fora different bitstream format.

One byte encodes a maximum frame rate as an 8-bit value.

One byte encodes frame resolutions, where each bit is a yes/no value fora different frame resolution.

One byte encodes a maximum bit rate as an 8-bit value were the value ofthe LSB is 2.

For example, the capabilities for all four video mode choices listedabove may be encoded as follows:

a Bitstream format byte=(00000011), where the LSB indicates that the IRVbitstream is supported and the next LSB indicates that the MRV bitstreamis supported,

Frame rate byte=(00001111) corresponding to 15 fps.

Frame resolution byte=(00000001), where the LSB indicates that the(160×120) frame resolution is supported and the next LSB indicates thatthe (320×240) frame resolution is not supported.

Bit rate byte=(01011111) corresponding to 200 kbps.

In yet another alternative embodiment, video negotiation is based oncombinations of bitstream format, maximum frame rate, and frameresolution, where the maximum bit rate is externally specified.

Referring now to FIG. 47, there is shown a flow diagram of the videonegotiation processing between two conferencing systems 100 (i.e., nodeA and node B) of FIG. 1. Node A sends a negotiation proposal to node B,where the proposal contains a set of capabilities encoded in the 32-bitcapabilities data structure as described above (step 4702). If theproposal corresponds to a unique video mode (i.e., the proposedcapabilities include only one bitstream format and only one frameresolution) and if that proposed video mode is acceptable to node B(step 4704), then the video negotiations have successfully chosen avideo mode acceptable to both nodes and node B sends the same proposalback to node A to accept the video mode for video conferencing (step4706).

If, however, the proposal is not a unique video mode (e.g., there ismore than one bitstream format and/or frame resolution) or if theproposal is not acceptable to node B (e.g., frame rate and/or bit rateis too high) (step 4704), node B determines whether it can make acounter proposal to node A (step 4708). A counter proposal is a subsetof the capabilities contained in the previous proposal (e.g., lowerframe rate or bit rate, fewer bitstream formats or frame resolutions).If node B does not have a counter proposal, then the video negotiationshave failed and node B sends node A a message rejecting the conference(step 4710).

If node B does have a counter proposal, then node B sends its counterproposal to node A (step 4712). Steps 4712-4720 are analogous to steps4702-4710 except that the roles of node A and node B are reversed. Thevideo negotiation processing of FIG. 47 continues until either amutually acceptable video mode is selected (i.e., successful videonegotiation) or nodes A and B are unable to identify a mutuallyacceptable video mode (i.e., unsuccessful video negotiation).

In an embodiment in which the allowable video modes are contained in atable in the video subsystem, it will be understood that the proposalsand counter proposals are constrained to being based on the specificmodes in the table.

In theory, the processing within a node for determining whether aproposed unique mode is acceptable and for generating a counter proposalto a particular proposal may depend on one or more of the followinginfluences:

The particular transport(s) that are available (e.g., LAN or ISDN).

The CPU processing bandwidth available for video processing.

The type of hardware installed in conferencing system 100 (e.g., thetype of video board 204 of FIG. 2 may influence which bitstream formatsare supported).

To a certain extent, these considerations are determined off line whenthe video subsystem table of mode choices is generated.

Video Manager Negotiation Support

Video manager 516 supports the following three interfaces to supportvideo negotiation.

VSystemAttrMap

DWORD VSystemAttrMap (DWORD, DWORD);

The VSystemAttrMap function takes a DWORD bit rate parameter and a DWORDflags parameter. The bit rate parameter is used by the caller toindicate the maximum communications bandwidth that can be used by avideo stream (unidirectional). Units are in kbits per second. The flagsparameter is for future use, and is undefined. VSystemAttrMap returnsDWORD bitmap used to code a range of video attributes. The returnedattributes bitmap is a function of the bit rate specified. For example,the attributes map may differ if the communications media is ISDN orLAN, where the former may allocate approximately 90 kbits for video andthe latter may allocate 150-300 kbits for video.

VDecodeAttrMap

VSTATUS VDecodeAttrMap (DWORD, LPVINFO);

The VDecodeAttrMap function decodes bitmap attributes into existingVINFO structure. VDecodeAttrMap takes DWORD bitmap which defines videoattributes and a VINFO structure. VDecodeAttrMap modifies (or builds)VINFO structure to reflect the DWORD bitmap defined attributes. TheVINFO fields wCaptureFormat, wMaxFrameRate, wWidth, and wHeight may bemodified as a result of VDecodeAttrMap. The VINFO structure is of typeLOCAL₋₋ STREAM. The decode produces a unique VINFO structure, and if theinput attributes map defines multiple options, then the best case(latest algorithm, highest frame rate, greatest capture resolution) ischosen.

VNegotiate

DWORD VNegotiate (DWORD, DWORD);

The VNegotiate function carries out a negotiation/comparison between twosystem attribute maps. VNegotiate takes a DWORD bitmap of a local systemand a DWORD bitmap of a remote system. VNegotiate returns a DWORD bitmapthat is the result of an internal video manager negotiation of videoattributes. If the returned value is zero, then negotiation failed. Ifthe returned value is non-zero, then negotiation was successful and thereturned value defines attributes common to the two systems. In orderthat repeated video negotiation is not required, if a non-zero value isreturned, then it represents a unique set of video attributes which canbe supported at both endpoints. This interface frees upper layersoftware (e.g., VCI 506) from interpreting and negotiating definedattribute bitmaps.

The following represents and example of the usage of these videonegotiation functions:

    __________________________________________________________________________    dwAttribtites                                                                       = VSystemAttrMap(DWORD)90, (DWORD)0);                                                              // ISDN call attempted.                            wVStatus                                                                            = VDecodeAttrMap(dwAttributes, &stVInfo);                               wVStatus                                                                            = VOpen(&stVInfo, &hVStrm, . . .);                                      wVStatus                                                                            = VCapture(hVStrm, ON);                                                 wVStatus                                                                            = VMonitor(hVStrm, ON);                                                 . . .                                                                         // At call establisbment VCI negotiation:                                     <"begin VCI negotiation" (produces dwRemoteAttributes)>                       dwNegotiatedAttributes = VNegotiate(dwAttributes, dwRemoteAttributes);        <"end VCI negotiation">                                                       if (|dwNegotiatedAttributes)                                                                // Common video attributes between endpoints could not be                     established. Audio/data call only?                              return("Failed negotiation -- Video cannot be established between             participants");                                                               }                                                                             if (dwNegotiatedAttributes|=dwAttributes)                                                        // Capture stream requires adjustment as a                                    result of negotiation.                                     {                                                                             // Rebuild VINFO structure to define new video.                               wVStatus  = VDecodeAttrMap(dwNegotiatedAttributes, &stVInfo);                 wVStatus  = VReOpen(hVStrm, &stVInfo, . . .);                                 wVStatus  = VCapture(hVStrm, ON);                                             wVStatus  = VMonitor(hVStrm, ON);                                             }                                                                             // Capture stream set correctly, call established, now link out video.        wVStatus = VLinkOut(hVStrm, . . .);                                           . . .                                                                         __________________________________________________________________________

Participant Information Exchange

The description for this section is the same as the description for thesection of the same name in U.S. Patent Application filed Nov. 15, 1994using U.S. Express Mail Label No. EG029471669.

Caller Attributes and Call Progress

The MakeConnection message contains 64 bytes of user-interpreted data(i.e., a 64-byte field that the user is free to define). A user may usesome or all of that field to identify certain attributes of the callerwhen the caller sends the MakeConnection message to a remote node(callee). These attributes may include the name of the user. In thiscase, the callee will know who is trying to place a conference callbefore the conference call is actually accepted by the callee. When thecallee is not operating in an auto-answer mode, the callee displays theuser-interpreted information contained in the MakeConnection message aspart of a dialog box that gives the user of the remote node the optionof accepting or rejecting the conference call based on that information.

When a caller attempts to place a conference call to a callee, theduration from the time that the user starts the process (e.g., selectinga "dial the selected phone number" option in a dialog box) until thetime that the attempt to place the conference call succeeds or fails maybe anywhere from a few seconds to as long as a minute. The caller useris presented with feedback to indicate the progress of the conferencecall during that duration. Similarly, the callee user is also presentedwith appropriate call-progress feedback.

The call-progress information presented to the caller user may includeaudible ringing or a display of simulated ringing (e.g., displaying agraphics image of a phone surrounded by vibration lines). When theattempt to place the conference call is successful, the caller notifiesthe user with an appropriate graphical display. Since a callee providesa reason code when rejecting a conference call, in that case, the callerdisplays the reason why the call was rejected to the user.

For the callee, the call-progress information may include a displaynotifying the callee that a connection has been requested. This displayincludes the name of the caller user when that information is providedin the user-interpreted data field. The callee also presents a graphicaldisplay to the user when the attempt to place the conference call iseither successful or unsuccessful, in which case, the caller's reasonfor rejecting is presented.

Referring now to FIG. 48, there is shown a flow diagram of thecall-progress processing when the placement of a conference call issuccessful. The call-progress processing of FIG. 48 may be summarized asfollows:

The caller user uses a dialog box to ask for a conference call to beplaced to the callee (step 4802).

The caller network stack puts a connection request packet on the networkto the callee (step 4804). It will be understood that steps 4802 and4804 include all the necessary inter-layer communications from thecaller conferencing application through to the caller network stack asshown in FIGS. 5 and 17.

The network stack informs the caller that the connection request wasplaced on the network (step 4806).

The caller presents ringing to the caller user (step 4808).

Some time after step 4804, the callee receives the connection requestpacket over the network (step 4810).

The callee sends an acknowledgment packet for the receipt of theconnection request packet over the network to the caller (step 4812).

Some time after step 4812, the caller receives the acknowledgment packetfrom the callee over the network (step 4814).

After step 4812, the callee informs the callee user that the caller istrying to place a conference call, where the callee identifies thecaller user to the callee user (step 4816).

The callee user selects the dialog box option to accept the conferencecall (step 4818).

The callee sends a connection accepted packet over the network to thecaller (step 4820).

The caller receives the connection accepted packet over the network fromthe callee (step 4822).

The caller informs the caller user that the connection has beenestablished (step 4824).

After step 4822, the caller sends a connection accepted packet back tothe callee over the network (step 4826).

The callee receives the connection accepted packet over the network fromthe caller (step 4828).

The callee informs the callee user that the connection has beenestablished (step 4830).

As shown in FIG. 48, the caller presents ringing to the user (step 4808)after it receives acknowledgment from the network stack that theconnection request packet has been placed on the network (step 4806). Inan alternative embodiment, the caller waits until after receiving thepacket acknowledging receipt of the connection request packet by thecallee (step 4814) before presenting ringing to the caller user.

If the callee rejects the conference call from the caller (instead ofstep 4818), then the callee sends a connection rejected packet to thecaller over the network (instead of step 4820). In this case, after thecaller receives the connection rejected packet (instead of step 4822),the caller informs the caller user that the conference call was rejectedand present the callee's reason for the rejection (instead of step4824). In this case, steps 4826, 4828, and 4830 are omitted.

Conference Participation Messages

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Flow Control Over Reliable Channels

Referring now to FIG. 36, there is shown a flow diagram of theprocessing of conferencing systems A and B of FIG. 1 to control the flowof signals over reliable channels. The description for this section isthe same as the description for the section of the same name in U.S.patent application Ser. No. 08/340,172, filed Nov. 15, 1994, nowabandoned.

Preemptive Priority-Based Transmission

Referring now to FIG. 37, there is shown a flow diagram of thepreemptive priority-based transmission processing implemented by thecommunications subsystem of conferencing system 100 of FIG. 1. Thedescription for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/340,172,filed Nov. 15, 1994, now abandoned.

Rate Negotiation

Referring now to FIGS. 38-41, there are shown state diagrams for therate negotiation processing. The description for this section is thesame as the description for the section of the same name in U.S. patentapplication Ser. No. 08/340,172, filed Nov. 15, 1994, now abandoned.

Dial Lists

The conferencing application presents to the user a dialog boxcontaining an alphabetized directory of possible callees forconferencing sessions. The user is able to initiate a conference call toone or more callees by selecting from the listed callees.

The conferencing systems support the ability to maintain and accessdifferent lists of possible callees. For example, there is a networklist maintained by a network administrator and a personal list for eachconferencing node. The network and personal lists may overlap (i.e.,share a subset of possible callees). The node user is able to access andedit (i.e., read/write/delete) her own personal list, but can onlyaccess (i.e., read only) the network list and the personal lists ofother nodes.

A node user is presented with the option of (1) accessing her personallist only or (2) accessing a combination (i.e., union) of her personallist and one other selected list (i.e., either the network list or thepersonal list of another node). When the combined list is selected, itis displayed as if it were a single alphabetized list with the calleesfrom the user's personal list identified in some fashion (e.g.,displayed in a different color)

Interrupt-Time Processing for Receiving Data Signals

As described earlier in this specification, video conferencing system100 of FIG. 5 is implemented under a Microsoft® Windows™ operatingsystem running on host processor 202 of conferencing system 100. Hostprocessor 202 of conferencing system 100 receives data signals from andtransmits data signals to hardware components that are external to hostprocessor 202. For example, during a LAN-based audio/video conference,host processor 202 receives compressed video signals from video board204 and compressed audio signals from audio/comm (ISDN) board 206, andtransmits these compressed audio and video signals to LAN board 210 fortransmission over the LAN to a remote conferencing system. Similarly,LAN board 210 transmits to host processor 202 the compressed audio andvideo signals which it receives over the LAN from the remoteconferencing system. Host processor 202 then transmits the compressedaudio signals to audio/comm (ISDN) board 206 for decompression andplayback. The compressed video signals are decompressed by hostprocessor 202.

Since host processor 202 is a serial processor, the various softwarecomponents that run on host processor 202 operate serially, that is,only one software component operates at a time. The operating system ofhost processor 202 controls the operations of the various softwarecomponents by establishing a task schedule by which the operating systemallocates the processing time of host processor 202 to the softwarecomponents.

In addition to being a serial processor, host processor 202 is also apreemptive processor. As a preemptive processor, the operating system isable to suspend the implementation of one function by host processor 202to have host processor 202 perform a second function. After the secondfunction has completed, the operating system causes host processor 202to resume the implementation of the first function from where it leftoff.

An external hardware component (e.g., video board 204, LAN board 210, oraudio/comm (ISDN) board 206) can ask the operating system to have hostprocessor 202 perform a particular function by sending host processor202 an interrupt signal. The operating system may cause the requestedfunction to be performed as a scheduled task of the normal taskschedule. In this case, the performance of the task is delayed until theoperating system schedules the task. Alternatively, the operating systemmay perform the requested function preemptively. That is, the operatingsystem may cause host processor 202 to suspend what it was doing longenough for host processor 202 to perform the requested function. In thiscase, the preemptive processing is said to be implemented duringinterrupt time.

For example, when LAN board 210 receives compressed audio and videosignals over the LAN from a remote conferencing system, LAN board sendshost processor 202 an interrupt signal to inform host processor 202 thatLAN board 210 has received data signals for host processor 202 toprocess. In order to provide high quality playback during audio/videoconferencing (especially for the audio signals), it is important toprocess the received data signals as quickly as possible. If hostprocessor 202 were to process the received data signals during anormally scheduled task of the operating system, then the resultingquality of the audio/video conference may be relatively low, since thereis no guarantee of a scheduled task executing within a reasonable periodof time under the Microsoft® Windows™ operating system.

The audio/video conferencing system processes the received audio andvideo signals preemptively during interrupt time. By processing thereceived data signals at interrupt time, the audio/video conferencingsystem is able to provide higher quality audio and video playback thanwould otherwise be provided if such processing were implemented as anormally scheduled operating system task.

In a possible embodiment of an audio/video conferencing system, theprocessing of the received audio and video signals could be attempted tobe implemented during the "receive" interrupt time that follows theinterrupt signal from LAN card 210 which informs host processor 202 thatLAN card 210 has received compressed audio and video signals for hostprocessor 202 to process. It has been discovered, however, that, underMicrosoft® Windows™ operating systems, when the audio/video conferencingsystem attempts to process completely the received data signals duringthe receive interrupt time, there may be undesirable results. Forexample, the operating system may crash (i.e., cease proper operations).

Referring now to FIG. 49, there is shown a representation of theinterrupt-time processing for receiving data signals by audio/videoconferencing system 100 of FIG. 5. When LAN board 210 sends hostprocessor 202 a receive interrupt signal (i.e., informing operatingsystem 4902 that LAN board 210 has received compressed audio/videosignals), operating system 4902 suspends the current processing of hostprocessor 202 and passes the interrupt signal to LAN comm stack 560. LANcomm stack 560 passes the interrupt signal to comm manager 518. Commmanager 518 causes the LAN comm software to read the received datasignals from LAN board 210. The LAN comm software passes the receiveddata signals to comm manager 518, which stores the received data signalsinto a queue in memory for subsequent processing. All of this processingoccurs during the receive interrupt time that follows the receiveinterrupt signal. The interrupt processing then terminates and theoperating system causes host processor 202 to resume the processing thathad previously been suspended.

Operating system 4902 receives regular clock interrupt signals (e.g.,every approximately 10 milliseconds). Operating system 4902 passes theseclock interrupt signals to comm manager 518. Comm manager 518 uses theclock interrupt time (which follows a clock interrupt signal) tocomplete the processing of any received audio and video signals thatwere queued into memory during the previous receive interrupt time. Thisclock-interrupt-time processing includes passing the received audio andvideo signals from the comm manager 518 to the transport independentinterface 510, which distributes the audio signals to the audio manager520 and the video signals to the video manager 516 for decompression andplayback processing. After the audio and video signals have beenprocessed, the interrupt processing terminates and the operating system4902 causes host processor 202 to resume the processing that hadpreviously been suspended.

It has been discovered that this strategy of completing the processingof the received data signals during the clock interrupt time avoids theproblems that may be associated with attempting to process completelythe received data signals during the receive interrupt time.

Interrupt-Time Processing for Transmitting Data Signals

During a conferencing session, conferencing system 100 generates audioand video signals for transmission to a remote conferencing system.These data signals become available for transmission in discrete packetsat discrete moments. That is, the data signals are not generated byconferencing system 100 in a perfectly steady stream. For example, whenvideo signals are generated at a rate of 10 frames per second, a set ofcompressed video signals corresponding to a video frame is generatedonce every 100 milliseconds. The set becomes available for transmissiononly after the entire frame has been compressed. Thus, video signalsbecome ready for transmission in discrete sets at discrete moments.

For typical video frames, each set of compressed video signals is brokenup into a plurality of data packets for transmission, for example, overthe LAN. If the conferencing system 100 attempted to transmit all of thedata packets for a video frame as soon as the data packets were ready,the LAN board would transmit the packets one right after the other withvery little time between packets and the remote conferencing system maynot be able to receive and process all of the data packets in such ashort period of time. As a result, some of the data packets may bedropped by the remote conferencing system causing the quality of theconferencing session to be adversely affected.

Referring now to FIG. 50, there is shown a representation of theinterrupt-time processing for transmitting data signals by audio/videoconferencing system 100 of FIG. 5. Rather than transmitting all of thedata packets as soon as they become available for transmission,conferencing system 100 uses the 10-msec clock interrupt signals(described in the previous section of this specification) to spread outthe packet transmission.

As shown in FIG. 50, video board 204 sends a "data-ready" interruptsignal to the host processor 202 to inform the operating system 4902that a set of compressed video signals corresponding to a video framehas been generated. The operating system 4902 passes this interruptsignal to the video manager 516. During the data-ready interrupt time,the video manager 516 reads the compressed video signals from the videoboard 204 and passes the compressed video signals to the transportindependent interface 510. TII 510 passes the compressed video signalsto the comm manager 518, which divides the video signals into packetsfor transmission over the LAN. Analogous processing is implemented forthe compressed audio signals generated by the audio/comm (ISDN) board206.

No packets are sent over the link to the remote conferencing systemduring the data-ready interrupt time. Rather, packets are sent duringthe interrupt times that follow the 10-msec clock interrupt signals.When the comm manager 518 receives a clock interrupt signal from theoperating system 4902, if there are one or more packets ready to besent, then the comm manager 518 sends a single packet to the LAN commsoftware 560 for transmission over the LAN to the remote conferencingsystem. The result is that the transmission of packets is spread overtime with at most one packet being transmitted every 10 msec.

By spreading the transmission of packets over time, the conferencingsystem 100 increases the probability that the remote conferencing systemwill have time to receive and process all of the packets. The result isimproved quality of the conferencing session due to the reduced numberof packet drops by the remote conferencing system.

Those skilled in the art will understand that other embodiments fallwithin the scope of the present invention. For example, the time periodbetween clock interrupt signals may be other than 10 msec. In addition,the maximum number of packets transmitted during any one clock interrupttime may be other than 1.

Auto Registration and Remote Confidence Testing

Auto registration is a process, along with appropriate mechanisms andutilities, for electronically registering a video conferencing product.The registration information is deposited in a customer support databasefor registered products. Auto registration is coupled with remoteconfidence testing, through which customers can test their ISDN line aswell as the basic functionality of their conferencing product.

A purpose of auto registration is to collect registration data for theproduct manufacturer. Auto registration is designed to increase thepercentage of customers who register their products by providing an easyand quick user interaction. The auto registration design allows forfuture changes and expansions.

Referring now to FIG. 51, there is shown a representation of autoregistration environment 5100 for video conferencing. The autoregistration environment 5100 comprises a new video conferencing node100 connected via network 110 to one more confidence test servers 5104.The confidence test servers 5104 are in turn connected via local areanetwork (LAN) 5106 to a network comprising an auto registration database(ARD) node 5110 and a customer support node 5108. It is assumed that,when network 110 is an ISDN network, the ISDN line is connected to newnode 100 and has been tested for functionality by the appropriateregional phone company.

New node 100 is operated by a customer who has purchased a videoconferencing product according to the present invention. New node 100comprises the following components/software packages:

A personal computer with an Intel® i486™/33 MHz processor or higher, VGAmonitor or better, and at least 8 megabytes of random access memory(RAM);

An audio/comm (ISDN) board such as audio/comm (ISDN) board 206 of FIG.2;

A LAN board such as LAN board 210 of FIG. 2;

A video board such as video board 204 of FIG. 2;

Video conferencing system software and video conferencing applicationsoftware, such as those shown in FIG. 5; and

Installation and diagnostics software.

Each confidence test server (CTS) 5104 is a high-speed personal computerwith an Intel® i486™ or better processor. The confidence test servers5104 provide three types of services:

(1) Accepts a registration form from new node 100 and deposits it intothe auto registration database of ARD node 5110.

(2) Plays an audio/video segment from a PC/VCR of CTS 5104 to test thebasic video conferencing functionality of new node 100. In this case,CTS 5104 behaves exactly like a remote node to the new node. That is,the video and audio streams go through the real-time video conferencingmechanisms that two live nodes would go through.

(3) Downloads an applet (e.g., answering machine software) to new node100 and installs it in the appropriate directory.

A typical configuration for confidence test server 5104 is as follows:

A personal computer with an Intel® i486™/33 MHz processor or higher, VGAmonitor or better, at least 8 megabytes of random access memory (RAM),at least 380 megabytes of hard drive;

An audio/comm (ISDN) board such as audio/comm (ISDN) board 206 of FIG.2;

A LAN board such as LAN board 210 of FIG. 2;

A video board such as video board 204 of FIG. 2;

A high-speed FAX/Modem 400E (e.g., to dial via a modem into the autoregistration database of ARD node 5110);

A 32-bit LAN adapter card;

A personal computer video cassette recorder (PC/VCR) connected to thehost processor through the serial port;

Video conferencing system software, such as that shown in FIG. 5; and

Microsoft® Windows for Workgroup™ or other appropriate software.

In an auto registration environment having more than one new node 100,all of the new nodes access the confidence test servers 5104 using thesame telephone number. In such an environment, when two or more newnodes 100 are simultaneously calling into two or more confidence testservers 5104, hub 5102 dispatches each incoming call to the nextavailable CTS 5104.

A PC/VCR is connected to the communications port of the host processorof each CTS 5104 via a serial line. The PC/VCR plays an audio/videosegment (up to approximately 30 seconds long) to the new node. The audioand video signals from the PC/VCR are input to the audio/comm (ISDN)board and the video board, respectively. In an alternative embodiment, aprogrammable laser disk is used instead of a PC/VCR. In anotheralternative embodiment, the confidence test server 5104 is able torecord the user of the new node in real time and play back the sessionto the user as part of the confidence testing. In yet anotheralternative embodiment, a live operator operates the confidence testserver 5104 to provide confidence testing services to the new nodes.

In another embodiment, the CTS consists of a PC that plays anaudio-video clip recorded in an AVI (audio-video-interleaved) file. TheAVI file is played on a PC running Microsoft® Video for Windows™software. A video capture board is used on this machine to generate NTSC(National Television Standards Committee) video signals. These videosignals are then fed as camera input to a CTS.

Auto registration database (ARD) node 5110 provides the autoregistration database into which the confidence test servers 5104deposit the completed registration forms. The confidence test servers5104 are connected to ARD node 5110 over a local area network, such asLAN 5106, and the registration forms are transmitted from CTS 5104 toARD node 5110 over the LAN. Alternatively, a modem may be used for theregistration form transfer. The registration forms are deposited intothe auto registration database of ARD node 5110 as a transaction toassure data integrity.

Referring now to FIG. 52, there is shown a representation of thearchitecture for auto registration and remote confidence testing for thenew node of FIG. 51. Auto registration software consists of a librarymodule that manages the registration user interface for all products andindependent communication modules that support different communicationmedia such as modem and ISDN. The user interface uses the services ofthe communication modules in a media-independent fashion through acommon communication interface. All the necessary information requiredto create dialog boxes is stored in two files: the ICOMM.INI file and aproduct-specific table.

As a step in initialization during setup, ICOMM.INI is created. Theregistration status for each product may be one of two states: SETUP andREGISTERED. On initialization, the status for each product is set toSETUP. The state SETUP is meant mainly for the case where the system isrebooted. Every time the product application is run, it will readICOMM.INI to check the registration status. In the case of the first runafter a reboot, the registration status will have the entry set toSETUP. This will tell the application that it has to bring up the dialogbox. Once the dialog box is invoked, the state SETUP has no moremeaning. The complete product status entry will be deleted in bothreboot and no reboot cases. Thereon, depending on the choice of the userto register or not, the registration status will be modified.

If the user decides to register and if the registration is successful,then the product status entry is created and its value is set toREGISTERED. If registration fails, then there is no action and theproduct status entry will not be present. In that case, the next time astandalone auto registration program is run, it will fail to find theproduct status entry in ICOMM.INI and thereby know that the product wasnot registered earlier.

Auto registration will be invoked at different times depending onwhether installation requires the system to be rebooted or not. If theproduct requires no additional hardware for installation, then autoregistration is called as the last step in the setup process. The user'schoice to register or not is recorded in ICOMM.INI. When the mainapplication is started again, ICOMM.INI will be read and if the productstatus entry is SETUP, the auto registration will be invoked again. Theuser's new choice will be recorded in ICOMM.INI and registration willproceed if required.

If the system needs to be rebooted after installing any hardware, thenauto registration will be called only by the main application and not bysetup. The application will read ICOMM.INI and find the product statusentry to be set to SETUP. In that case, the application will prompt theuser to register. The user's new choice will be recorded in ICOMM.INIand registration will proceed if required.

Referring now to FIG. 53, there is shown a flow diagram of theprocessing for the auto registration and remote confidence testing ofauto registration environment 5100 of FIG. 51. This processing isimplemented as a client/server application, where the client is a newnode 100 and the server is a confidence test server 5104.

New node 100 places a call to connect to a CTS 5104 (step 5302 of FIG.53) and transmits a registration record (step 5304). In response, CTS5104 runs the remote confidence test (step 5306). Depending upon theresults of the remote confidence test, new node 100 sends CTS 5104 anappropriate acknowledgement/failure message (step 5308). CTS 5104 maythen download a free applet onto new node 100 (step 5310), whichinstalls the applet (step 5312). A purpose of the free applet is toencourage new users to register their nodes. After CTS 5104 disconnects(step 5314), CTS 5104 deposits the registration record into the autoregistration database of ARD node 5110 (step 5316) and reports anyfailures to customer support node 5108 (step 5318).

The processing shown in FIG. 53 may be altered to cover a number ofpossible alternatives for auto registration, such as:

Auto registration only (i.e., no incentives);

Auto registration+confidence testing;

Auto registration+free applet; and

Auto registration+confidence testing+free applet.

Referring now to FIG. 54, there is shown a flow diagram of theprocessing implemented by the client (i.e., new node 100) for the autoregistration processing of FIG. 53. As one of the last steps ininstallation (or re-installation) and diagnostics for new node 100, adialog box is displayed to inform the user that he/she may complete theinstallation by implementing remote confidence testing of the productand registering the product at the same time (step 5402 of FIG. 54).Alternatively, auto registration may be invoked by the main applicationafter the point where the detection of the presence of hardware anddrivers is found to be successful. If the user does not want to registerthe new node 100 and selects the "Cancel" option (step 5404), then theproduct registration status variable Auto₋₋ Reg₋₋ State is set to thevalue NOT₋₋ REGISTERED (step 5406) and the auto registration processingis terminated.

If the user does want to register the new node 100 and the "OK" optionis selected, then a registration record is prepared (step 5408). Aregistration record contains user information, extended userinformation, communications settings, communication port settings, thephone/ISDN number, the product serial number, username/password forserver, and the date that the registration record is prepared. Theinformation is stored in the ICOMM.INI file.

User information includes the name, address, phone number, facsimilenumber, and position of the user. User information is common for theregistration of all products of that user. Extended user information, onthe other hand, is information that may vary from product to product andincludes answers to specific questions posed to the user about theparticular product. Extended user information is stored in a separatetable for each product.

Communications settings information is the source of informationrequired to set up communications. In the case of modem, it includesport, baud rate, word length, start bits, stop bits, and parity.Communications port settings provides auto registration with informationas to how the communications ports are mapped to the PC interruptrequest levels. The phone/ISDN number is the server phone number or ISDNnumber that is to be dialed in order to transmit the registrationrecord.

The product serial number may be stored either on the installationdiskettes or in a word in the EEPROM. During setup, the setup programobtains the serial number from the appropriate source and writes itunder a SERIAL₋₋ NUMBER key in the ICOMM.INI file. Since the serialnumber in ICOMM.INI may get corrupted, if the user decides to registerthe product at a time other than during the initial setup, instead ofreading the serial number form the ICOMM.INI file, the application isrequested to read the serial number form the appropriate source andwrite it to ICOMM.INI. In other words, the serial number is buffered inICOMM.INI for auto registration to read.

After the registration record is prepared, the user initiates a videoconference call from the new node 100 to confidence test server 5104,using a designated toll-free number (step 5408). If the call is notcompleted (either because the line is busy or because of a line problem(step 5410), then the product registration status variable Auto₋₋ Reg₋₋State is set to the value NOT₋₋ REGISTERED (step 5406) and the autoregistration processing is terminated.

If the connection to CTS 5104 is established, then the new node 100sends the registration record to CTS 5104, sets the product registrationstatus variable Auto₋₋ Reg₋₋ State to the value IS₋₋ REGISTERED, setsthe confidence test timer, and waits for remote confidence testing (step5412). If the confidence timer times out without receiving any messagesor active streams from CTS 5104 (step 5414), then a failure message isdisplayed (step 5416) and the auto registration processing continues tostep 5420.

Otherwise, the new node 100 receives and renders an audio/video clipfrom CTS 5104 (step 5418). Dialog boxes are then displayed for the userto report to CTS 5104 the audio and video status of the clip and to savethe confidence test results (step 5420). The new node 100 then receivesand installs the free applet from the CTS 5104 (step 5422). The new nodeterminates the auto registration processing by disconnecting from theCTS 5104 (step 5424).

Referring now to FIG. 55, there is shown a flow diagram of theprocessing implemented by a confidence test server 5104 for the autoregistration processing of FIG. 53. The CTS 5104 answers the call fromthe new node 100 and stores the caller's ISDN number (step 5502 of FIG.55). The CTS 5104 accepts a registration record from the new node 100and sends an acknowledgement message to the new node 100 (step 5504).The CTS 5104 starts the confidence testing process by sending a messageto the new node 100 to display the proper instructions to the user(e.g., "Make sure your head set is connected.") (step 5506). The CTS5104 then transmits the audio/video clip from its PC/VCR to the new node100 (step 5508).

After the audio/video clip is over, the CTS 5104 a message instructingthe new node 100 to question the user for the audio and video status ofthe clip (step 5510). If the messages returned from the new node 100indicate there was a failure in either the audio or video streams (step5512), then the CTS 5104 sets the CTS₋₋ State variable to theappropriate value (i.e., AUDIO₋₋ FAILURE, VIDEO₋₋ FAILURE, or BOTH₋₋FAILED), and prepares and sends a message to the customer support node5108 (step 5514). If there were no failures, then the confidence testwas passed and the CTS₋₋ Status variable is set to the value CTS₋₋PASSED (step 5516). The free applet is then downloaded to the new node100 and a message is sent to instruct the new node 100 to hang up (step5518). After the line is disconnected (step 5520), the CTS 5104 preparesan auto registration database transaction and sends the ARD transactionto ARD node 5110 (step 5522). The ARD transaction comprises theregistration record, the new node's ISDN number, and the confidence testresults.

The unique product serial number is stored in a word in the EEPROM ofthe product hardware. Alternatively, the product serial number may bestored in the installation disks that are shipped with the videoconferencing product. In the latter case, the number is then saved in alocation in the EEPROM on the audio/comm (ISDN) board.

The product auto registration status variable Auto₋₋ Reg₋₋ State has oneof two values: NOT₋₋ REGISTERED and IS₋₋ REGISTERED. These values arecoded in a word in the EEPROM on the audio/comm (ISDN) board.

The CTS status variable CTS₋₋ State stores the results of the confidencetesting and has one of four values: AUDIO₋₋ FAILURE, VIDEO₋₋ FAILURE,BOTH₋₋ FAILURE, and CTS₋₋ PASSED. The values are coded in a word in theEEPROM on the audio/comm (ISDN) board.

The auto registration and remote confidence testing procedures may berepeated periodically to register and/or validate system upgrades orupdates.

Referring now to FIG. 56, there is shown a representation of the autoregistration file format. The auto registration module creates the file"reg.tmp" in the Microsoft Windows™ directory to store user datacollected from the main auto registration dialog. The reg.tmp file is atemporary file and is deleted when the user exits the auto registrationprogram. The reg.tmp file is transferred from the new node 100 of FIG.51 to a confidence test server 5104 via modem using X-modem protocolwhen successful connection is made.

The reg.tmp file comprises a file header followed by blocks ofregistration data for every product that is being registered in aparticular file transfer. The registration data comprises userinformation and survey data. If there is no survey dialog for aparticular product or if the user does not make any selections in theoptional survey, then that block of optional data will not be part ofthe registration file.

In alternative embodiments of auto registration, automatic softwareupgrades are provided through the infrastructure of the autoregistration and remote confidence testing services described in thissection. In other alternative embodiments, use of the ISDN board isdisabled until the video conferencing product has been registered usingthe auto registration described in this section. The fields of thereg.tmp file represented in FIG. 56 are described in Appendix K of thisspecification. Appendix K also describes the data structures for autoregistration.

Alternative Embodiments

The description for this section is the same as the description for thesection of the same name in U.S. patent application Ser. No. 08/157,694.

In the embodiment of FIG. 2, conferencing system 100 comprises threeseparate boards: video board 204, audio/comm (ISDN) board 206, and LANboard 210. Those skilled in the art will understand that alternativeembodiments of the present invention may comprise other boardconfigurations. For example, an alternative embodiment may comprises acombined video/audio board (for performing video capture and audiocapture and playback), an ISDN board (for transmitting and receivingdata over an ISDN network), and a LAN board (for transmitting andreceiving data over a LAN network).

It will be further understood that various changes in the details,materials, and arrangements of the parts which have been described andillustrated in order to explain the nature of this invention may be madeby those skilled in the art without departing from the principle andscope of the invention as expressed in the following claims. ##SPC1##

What is claimed is:
 1. A computer-implemented process for videoconferencing, comprising the steps of:(a) negotiating video capabilitiesby a first node with a second node, wherein the first node transmits afirst video capabilities signal comprising two or more fields each fieldcomprising one or more bits and each field corresponding to a differentset of video capabilities for the first node; (b) selecting one set ofvideo capabilities for the first node for video conferencing between thefirst and second nodes; and (c) performing video conferencing by thefirst node with the second node in accordance with the selected set ofvideo capabilities.
 2. The process of claim 1, wherein step (a)comprises the steps of:(1) generating the first video capabilitiessignal by the first node based on a first-node table that identifies oneor more sets of video capabilities for the first node; (2) transmittingthe first video capabilities signal from the first node to the secondnode, wherein the second node generates a second video capabilitiessignal in response to the first video capabilities signal and transmitsthe second video capabilities signal to the first node; (3) receivingthe second video capabilities signal by the first node from the secondnode; (4) comparing the second video capabilities signal with the firstvideo capabilities signal by the first node to determine if videocapabilities negotiation is complete; and (5) continuing negotiatingvideo capabilities if negotiation is not complete.
 3. The process ofclaim 2, wherein each valid bit of the first video capabilities signalcorresponds to a different set of video capabilities in the first-nodetable.
 4. The process of claim 3, wherein each set of video capabilitiesin the first-node table identifies a frame resolution and a frame rate.5. The process of claim 4, wherein each set of video capabilities in thefirst-node table further identifies a bitstream format and a bit rate.6. The process of claim 3, wherein the first-node table identifiesrelative preferences of the sets of video capabilities.
 7. The processof claim 2, wherein the first-node table identifies relative preferencesof the sets of video capabilities.
 8. The process of claim 1, whereineach set of video capabilities identifies a frame resolution and a framerate.
 9. The process of claim 8, wherein each set of video capabilitiesfurther identifies a bitstream format and a bit rate.
 10. An apparatusfor video conferencing, comprising:(a) means for negotiating videocapabilities by a first node with a second node, wherein the first nodeis capable of transmitting a first video capabilities signal comprisingtwo or more fields, each field comprising one or more bits and eachfield corresponding to a different set of video capabilities for thefirst node; (b) means for selecting one set of video capabilities forthe first node for video conferencing between the first and secondnodes; and (c) means for performing video conferencing by the first nodewith the second node in accordance with the selected set of videocapabilities.
 11. The apparatus of claim 10, wherein means (a) iscapable of:(1) generating the first video capabilities signal by thefirst node based on a first-node table that identifies one or more setsof video capabilities for the first node; (2) transmitting the firstvideo capabilities signal from the first node to the second node,wherein the second node is capable of generating a second videocapabilities signal in response to the first video capabilities signaland transmitting the second video capabilities signal to the first node;(3) receiving the second video capabilities signal by the first nodefrom the second node; (4) comparing the second video capabilities signalwith the first video capabilities signal by the first node to determineif video capabilities negotiation is complete; and (5) continuingnegotiating video capabilities if negotiation is not complete.
 12. Theapparatus of claim 11, wherein each valid bit of the first videocapabilities signal corresponds to a different set of video capabilitiesin the first-node table.
 13. The apparatus of claim 12, wherein each setof video capabilities in the first-node table identifies a frameresolution and a frame rate.
 14. The apparatus of claim 13, wherein eachset of video capabilities in the first-node table further identifies abitstream format and a bit rate.
 15. The apparatus of claim 12, whereinthe first-node table identifies relative preferences of the sets ofvideo capabilities.
 16. The apparatus of claim 11, wherein thefirst-node table identifies relative preferences of the sets of videocapabilities.
 17. The apparatus of claim 10, wherein each set of videocapabilities identifies a frame resolution and a frame rate.
 18. Theapparatus of claim 17, wherein each set of video capabilities furtheridentifies a bitstream format and a bit rate.
 19. A first videoconferencing node, comprising:a conferencing subsystem; a communications(comm) subsystem; and a video subsystem, wherein:the conferencingsubsystem is capable of negotiating video capabilities with a secondnode; the comm subsystem is capable of transmitting a first videocapabilities signal comprising two or more fields, each field comprisingone or more bits and each field corresponding to a different set ofvideo capabilities for the video subsystem; the conferencing subsystemis capable of selecting one set of video capabilities for the videosubsystem for video conferencing between the first and second nodes; andthe video subsystem is capable of performing video conferencing with thesecond node in accordance with the selected set of video capabilities.20. The apparatus of claim 19, wherein:the conferencing subsystem iscapable of generating the first video capabilities signal based on afirst-node table that identifies one or more sets of video capabilitiesfor the video subsystem; the comm subsystem is capable of transmittingthe first video capabilities signal to the second node, wherein thesecond node is capable of generating a second video capabilities signalin response to the first video capabilities signal and transmitting thesecond video capabilities signal to the comm subsystem; the commsubsystem is capable of receiving the second video capabilities signalfrom the second node; the conferencing subsystem is capable of comparingthe second video capabilities signal with the first video capabilitiessignal to determine if video capabilities negotiation is complete; andthe conferencing subsystem is capable of continuing negotiating videocapabilities if negotiation is not complete.
 21. The apparatus of claim20, wherein each valid bit of the first video capabilities signalcorresponds to a different set of video capabilities in the first-nodetable.
 22. The apparatus of claim 21, wherein each set of videocapabilities in the first-node table identifies a frame resolution, aframe rate, a bitstream format, and a bit rate.
 23. The apparatus ofclaim 20, wherein the first-node table identifies relative preferencesof the sets of video capabilities.
 24. The apparatus of claim 19,wherein each set of video capabilities identifies a frame resolution, aframe rate, a bitstream format, and a bit rate.